End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

Enterprise track: Step Functions/EventBridge + Glue + data quality on top of the v1 serverless ELT

🌟 Build a production-lite serverless ELT pipeline on AWS, enabling efficient data ingestion and transformation from S3 to Parquet with minimal overhead.

Powering Data Dreams: From Orchestration to Analytics with Cloud Precision

An automated end-to-end data pipeline using Apache Airflow, Spark, and MinIO for processing NYC Taxi datasets. Features containerized infrastructure (Docker), distributed transformations, and data quality assurance with Great Expectations.