New Features and Orchestration

The purpose of these exercises is to become familiar with some updated DataWORQ features, as well as flow orchestration.

Exercises

0. You are the manager tracking sales for stores from Flat File Management. Given three flat files with sales data per store (DailySales_Store1.csv, DailySales_Store2.csv and DailySales_Store3.csv), you want to build on the former solution:

1. You now want to stage the input data first in a database, and then produce a table that aggregates sales data ("Quantity", "Revenue") at the total store level.

2. Change the above solution to use parameterized table names. Use these in the SQL query.

3. Change the above solution to use a minimum of 4 flows, orchestrated by a master flow.

4. Improve the total amount of time it takes to perform the initial data load, if possible.

5. Schedule the load of DailySales_Store1.csv and DailySales_Store2.csv to happen nightly at midnight, and the rest of the data loads/processing to happen weekly.

6. Notify yourself when the above successfully happens.

Hints

1. Read through Flat File Source, Derive, Filter, Flat File Export, Database Source, Database Export, and Flow Connections.

2. Read through Database Source.

3. Read through Flow.

4. Read through Run Profiles.

5. Read through Run Profiles.

6. Read through Email Alerting.