The purpose of these exercises is to become familiar with some updated DataWORQ features, as well as flow orchestration.
Exercises
- You are the manager tracking sales for stores from Flat File Management. Given three flat files with sales data per store (DailySales_Store1.csv, DailySales_Store2.csv and DailySales_Store3.csv), you want to build on the former solution:
- You now want to stage the input data first in a database, and then produce a table that aggregates sales data ("Quantity", "Revenue") at the total store level.
- Change the above solution to use parameterized table names. Use these in the SQL query.
- Change the above solution to use a minimum of 4 flows, orchestrated by a master flow.
- Improve the total amount of time it takes to perform the initial data load, if possible.
- Schedule the load of DailySales_Store1.csv and DailySales_Store2.csv to happen nightly at midnight, and the rest of the data loads/processing to happen weekly.
- Notify yourself when the above successfully happens.
Hints
- Read through Flat File Source, Derive, Filter, Flat File Export, Database Source, Database Export, and Flow Connections.
- Read through Database Source.
- Read through Flow.
- Read through Run Profiles.
- Read through Run Profiles.
- Read through Email Alerting.