Rahul Gupta

Data Engineer building large-scale batch and streaming data platforms on AWS and GCP. Designs end-to-end pipelines in Python, SQL and Apache Airflow processing billions of events daily, powering real-time decisioning, ML models and business analytics. Experience spans warehouse architecture (BigQuery, Redshift, Druid), enterprise cloud migration and productionizing LLM workloads, with a track record of cutting infrastructure costs 30–35% at petabyte scale.

Core stack: Python · SQL · Apache Airflow · BigQuery · Redshift · Apache Druid · Apache Superset · dbt · Terraform · Docker · GKE · GitHub Actions

All authors

All authors

Blogs

Prep data for AI vs. Fabric Data Agent Instructions: What Goes Where?

By

Making Data AI-Ready, Part 3 (This is the final article in a three-part series...

Keeping Track of my ConEmu CLIs

By

I use ConEmu for my terminal interface. I’m still on Windows 10 at home,...

10 Decisions to Make Before You Create A Fabric Workspace

By

Creating a Fabric workspace takes about 30 seconds. Restructuring workspaces after people have built...

Read the latest Blogs

Forums

A design question in a parent child relationship

By eichnerm

I am not sure which is the best way to proceed. I have parent...

Hyperscale Replicas III

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Hyperscale Replicas III

Development Is Fun

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Development Is Fun

Visit the forum

Question of the Day

Hyperscale Replicas III

In an Azure SQL Database Hyperscale Edition, how many named replicas can be configured?

See possible answers