/
High Level Design For Mapping Non Esb Orders

High Level Design For Mapping Non Esb Orders

Owned by Yuvraj Darekar
Oct 10, 2024
2 min read

Components and Flow:

  1. Cron Job:

    • Purpose: Triggers the daily job to fetch non-Eshop orders from the channel_payment_report (CPR) table.

    • Trigger Time: Configured to run at a set time each day (e.g., midnight).

    • Actor: AppEngine Service

  2. Data Retrieval:

    • The cron job will fetch all records from the channel_payment_report where order_item_id is empty or null.

    • These non-Eshop orders are identified using the order_id and sku fields.

  3. Order Mapping Logic:

    • Service Layer (AppEngine):

      • The fetched non-Eshop orders are mapped to the sale_order_report_v2 BigQuery table using the order_id (mapped to customer_order_id) and sku columns.

      • The system will retrieve relevant order details, including missing information like order_item_id and other fields.

      • The mapping process will match and fill missing data for orders.

  4. Data Validation:

    • The service will validate that the mapped data is accurate and matches the expected structure.

    • If there are any unmatched records, errors will be logged via SLF4J for further analysis.

  5. Data Update:

    • Delete Operation: Once the mapping is complete, the matched records will be deleted from the CPR table (non-Eshop orders) to clean up the table.

    • Insert Operation: The mapped data will then be inserted back into the channel_payment_report table.

  6. Error Logging:

    • Any exceptions or errors encountered during the process, such as unmatched orders or database issues, will be logged using SLF4J.

  7. Database Layer:

    • The system interacts with the BigQuery sale_order_report_v2 table and the SQL channel_payment_report table for data retrieval and updates.

Key Technologies Involved:

  • Google AppEngine Service: The main service handling the business logic.

  • BigQuery: To store and retrieve order details.

  • SQL Database (CPR table): Where the payment details and non-Eshop orders are stored and updated.

  • SLF4J: For logging errors and exceptions.

HLD Diagram (Description):

  • Cron Job (AppEngine): Starts the daily process of fetching non-Eshop orders.

  • Order Mapping Logic: Maps the fetched data from CPR to BigQuery.

  • Data Validation: Ensures that the mapping is correct.

  • Data Update: Deletes the non-Eshop orders from CPR and inserts the mapped data.

  • Error Logging (SLF4J): Logs any issues during the process.

Related content