End-to-end data engineering project answering the question:
“How much revenue did we generate, and how is it distributed across products, customers, order statuses, and geography?”
Built in Google Colab with PySpark using a Medallion (Raw → Bronze → Silver → Gold) architecture, SCD, surrogate keys, a Date Dimension,
and Upserts into the Gold layer, plus seven business visualizations.
Key Features:
• Medallion data pipelines: /Raw, /Bronze, /Silver, /Gold
• SCD implementation (Type 2) for changing dimensions, surrogate/business keys management
• Date Dimension table for robust time-based analysis
• Upsert logic writing curated facts/dimensions into Gold
• Seven required plots: Revenue by Category (all), Top 10 Subcategories, Top 10 Customers, Revenue by Order Status (all), Top 10 Countries, States, Cities
• Code efficiency, import utilities, and clear, commented steps with DataFrame previews
The result is a production-style analytics layer that delivers trusted revenue insights across products, customers, order status, and geography.