Specification for em-processor-fn lambda function

This technical document outlines the S3 tree structure and the functionality for the lambda function {bztest | bzprod}-em-processor-fn.

This function automates the process of loading data from an Excel file into the Device MFG table devicemfg_info.

S3 Tree Structure

The S3 bucket used for this process is named bluvision-bluzone-us-east-1.

Within this bucket, the following folder structure is utilized:

The {bztest | bzprod} part of the path represents the environment, either the test environment (bztest) or the production environment (bzprod).

The {manual-processing} folder is used for storing seed files that require manual processing.

The {year}/{month}/{day} folders are used for organizing the seed files that were automatically uploaded by other lambda functions, for example, {bztest | bzprod|-ftp-loader-fn.

Process and Functionality

1. Seed File Placement: An operator of Bluzone places a seed file that meets the required specifications in the monitored S3 path structure under the appropriate folder /{manual-processing}` for a particular environment (bztest or bzprod).

2. Seed File Placement: Other lambda functions automatically upload files under the directory /emmicro/{bztest | bzprod}/{year}/{month}/{day}

3. Lambda Function Triggering: The placement of the seed file in the monitored S3 path structure triggers the execution of the lambda function {bztest | bzprod}-em-processor-fn.

4. Data Loading: The lambda function reads the Excel file from the S3 path and extracts the relevant data.

5. Data Insertion: The extracted data is then inserted into the Device MFG table devicemfg_info in the corresponding environment (test or production) by calling an API in portal.

6. Completion: After the data is successfully loaded into the devicemfg_info table, the process is considered complete.

Technical Diagram

Open

Use Case

The primary use case for this automated process is to enable Bluzone operators to easily update the Device MFG table devicemfg_info by simply placing a seed file that meets the required specifications in the designated S3 path structure.

This process eliminates the need for manual intervention in loading the data from the seed file into the table, saving time and reducing the possibility of human errors.

By organizing the processed seed files in a date-based folder structure, the system maintains a historical record of the files that have been processed, which can be useful for auditing, debugging, and tracking purposes.

Excel File Structure

Attached you can find a picture and sample file to illustrate the structure/format of the Excel file used as the seed file.

Open

Open

Conclusion

The {bztest | bzprod}-em-processor-fn lambda function, along with the defined S3 tree structure, provides an efficient and automated solution for loading data from Excel files into the Device MFG table devicemfg_info.

This process streamlines the data loading workflow for Bluzone operators, ensuring data consistency and reducing manual effort.