← Portfolio

Sales data product: from CSV to star schema

MedallionApache AirflowdbtSparkDelta LakeClickHouseKimballSuperset

Medallion layers, one Airflow DAG

Airflow DAG northwind_pipeline with the tasks landing, bronze, silver, gold and consumption

Landing zone

Bronze

Silver

Gold

Consumption

Silver: business objects

Silver data model: customer, sales_order, sales_order_line, product, product_category, supplier, employee, shipper

Gold: dimensions and facts

Gold star schema: fact_sales_order_line and fact_sales_order with the dimensions date, customer, product, employee and shipper

The result in Superset

$1,265,793
net revenue 1996–1998
830
orders
$1,525
average order value
37
orders shipped late

Quick checks