Stop Using Pandas for Aggregations — Try DuckDB Instead

If you've ever loaded a 2 GB CSV into pandas just to run a few aggregations — and watched your machine struggle — there's a better tool for the...

Optimizing Redshift Performance by Configuring WLM Queues

Efficient query performance in Amazon Redshift often comes down to how well you manage workload concurrency. Redshift's Workload Management (WLM) queues enable you to control how queries share resources,...

Data Engineering Books Worth Having on Your Shelf (or your tablet)

Good documentation gets you started. Good books get you deep. After years of working with cloud data platforms, SQL engines, and machine learning pipelines, a handful of titles keep...

Quick Tip: Check Most DTU Expensive Queries in Azure SQL Database

Optimizing Azure SQL Database performance often begins with identifying the most resource-intensive queries. Understanding which queries consume the most DTU (Database Transaction Unit) helps prioritize tuning efforts and improve...

PlanTrace Now Supports PostgreSQL

PlanTrace Now Supports PostgreSQL

The same plan analysis you know from Redshift, now for PostgreSQL — including EXPLAIN ANALYZE, buffer stats, and JSON format.

When...

PlanTrace: Stop Reading Redshift EXPLAIN Plans. Start Seeing Them

PlanTrace: Stop Reading Redshift EXPLAIN Plans. Start Seeing Them

Introducing PlanTrace — a free, browser-based tool that turns raw execution plans into interactive graphs and actionable tuning insights.

plantrace.studyyourdata.com

PlanTrace...

AWS RDS Aurora - our first approach

When starting with AWS RDS Aurora for managing relational databases in the cloud, many data engineers face the challenge of leveraging its cutting-edge features while maintaining performance and reliability....

How to Find Expensive Queries in Amazon Redshift

Slow-running queries can degrade your Redshift cluster’s performance and lead to increased costs. Identifying the most expensive queries is crucial to optimize resource usage and improve overall system efficiency.

Step-by-Step...

Cloud Databases Made Simple: Creating Your First AWS RDS Instance

I recently started full time learning of Amazon Web Services (AWS). I found that AWS's Relational Database Service
(RDS) is really user-friendly. RDS simplifies many of the
complicated aspects of...

Introduction to IaC: Deploying Data infrastructure to Azure using Terraform (Part 4 - Using Github)

 In Part 1, we explain the basics and prerrequisites to work with Terraform.In Part 2, we created our first Azure resource.In Part 3, we deployed our fist solution (Azure...