In this repository I have developed ETL data pipeline using GCP services like GCS, Big Query, Dataproc, Cloud composer(Airflow) and also setting up GCP resources this repository is python 7 sql based, code is parameterized, so values like, path's, credentials are not hardcoded, also demonstrating few patterns used in production like pipelines.

Google data analytics professional certificate capstone project

Code/Homework/Notes from DE Zoomcamp 2024

Capstone project for Google Data Analytics Course

A basic implementation of hadoop cluster using GCloud DataProc

This project analyzes a technology product sales dataset to derive business insights, enhance data analysis skills, & data visualization. Key steps include EDA & Pre-Processing, analysis, and visualization, leading to strategic recommendations for improving sales performance.

End-to-end wildfire intelligence data platform using Airflow, Spark, BigQuery, dbt, and Tableau for real-time risk monitoring and decision support.

This project analyzes 19M+ flight records (sourced from Kaggle in Parquet format) to uncover trends in delays, taxi times, and cancellations—driving insights for improved airline scheduling and operational efficiency.

ELT pipeline using Apache Airflow to scrape property data from www.properstar.com, load it into BigQuery, and transform the data with SQL.