Killing Explicit Sort Steps in Redshift with the Right SORTKEY

You paste a slow Redshift query plan into PlanTrace and one of the tuning insights reads "SORTKEY optimization candidate", pointing at an explicit sort step that's adding cost you...

5 SQL Tricks Worth Remembering Before You Close the Laptop

Disclosure: this post may contain links to books as an affiliate link. If you purchase through it, this site may earn a small commission at no extra cost to...

Stop Using Pandas for Aggregations — Try DuckDB Instead

If you've ever loaded a 2 GB CSV into pandas just to run a few aggregations — and watched your machine struggle — there's a better tool for the...

Optimizing Redshift Performance by Configuring WLM Queues

Efficient query performance in Amazon Redshift often comes down to how well you manage workload concurrency. Redshift's Workload Management (WLM) queues enable you to control how queries share resources,...

PostgreSQL 18 Finally Makes BUFFERS the Default. Here Is Why That Matters

You run EXPLAIN ANALYZE on a slow query, stare at the plan, and something still feels off. The estimated rows look reasonable, the node timings add up, but you...

AI and Data Engineering on the Edge: The Good, the Bad, and the Overhyped

Running AI and data pipelines on the edge instead of the cloud has gone from a niche embedded concern to a default option on a lot of architecture diagrams....

PostgreSQL: Diagnosing and Fixing a Sequential Scan with No Filter Predicate

A sequential scan is not always a problem — PostgreSQL's planner often chooses one correctly for small tables or queries that return most of a table's rows. The dangerous...

AWS RDS Aurora - our first approach

When starting with AWS RDS Aurora for managing relational databases in the cloud, many data engineers face the challenge of leveraging its cutting-edge features while maintaining performance and reliability....

Data Engineering Books Worth Having on Your Shelf (or your tablet)

Good documentation gets you started. Good books get you deep. After years of working with cloud data platforms, SQL engines, and machine learning pipelines, a handful of titles keep...

How to Find Expensive Queries in Amazon Redshift

Slow-running queries can degrade your Redshift cluster’s performance and lead to increased costs. Identifying the most expensive queries is crucial to optimize resource usage and improve overall system efficiency.

Step-by-Step...