ETL Antipattern: Lazy Metadata
If data is a train, then metadata is the track on which it travels. A good metadata definition in ETL processes will help to ensure that the flow of...
2020-12-15
19 reads
Tim Mitchell
- Interests: SQL Server, Data Warehousing, ETL, Data Architecture, Python, Dbt
- Website: https://www.TimMitchell.net
- Bluesky: tmitch.net
ETL Antipattern: Performing Full Loads Instead of Incremental Loads
In my last post in the ETL Antipatterns series, I wrote about the common antipattern of ingesting or loading more data than necessary. This brief post covers one specific...
2020-12-14
41 reads
ETL Antipattern: Processing Too Much Data
In my continuing series on ETL Antipatterns, I’ll discuss the problem of loading too much data in extract-transform-load processes. ETL Antipattern: processing too much data A common design flaw...
2020-12-13
15 reads
ETL Antipattern: Start With Writing Code
In this first post in my series on ETL Antipatterns, I’m going to discuss one of the most common missteps when building an extract-transform-load (ETL) process: jumping straight into...
2020-12-12
30 reads
The Eleven Days of Festivus 2020
We’re rounding the corner to the second half of December, which means it’s time for my favorite holiday: Festivus! Like many of you, I enjoy gathering around the Festivus...
2020-12-12
7 reads
Creating Your First Azure Data Factory
Azure Data Factory has grown in both popularity and utility in the past several years. It has evolved beyond its significant limitations in its initial version, and is quickly...
2020-09-08 (first published: 2020-08-28)
506 reads
My New Favorite Demo Dataset: Dunder Mifflin Data
Those of us who write technical articles and deliver technical presentations are always on the lookout for the perfect data set for demonstration and testing. Microsoft has done a...
2020-08-17 (first published: 2020-08-06)
661 reads
The What, Why, When, and How of Incremental Loads
When moving data in an extraction, transformation, and loading (ETL) process, the most efficient design pattern is to touch only the data you must, copying just the data that...
2020-08-05 (first published: 2020-07-23)
561 reads
Reusing a Recordset in an SSIS Object Variable
A few years back, I wrote a blog post about using an SSIS object variable as a data flow source. In that post, I described how you could load...
2020-07-30
258 reads
Comparing SSIS and Azure Data Factory
For the better part of 15 years, SQL Server Integration Services (SSIS) has been the go-to enterprise extract-transform-load (ETL) tool for shops running on Microsoft SQL Server. More recently,...
2020-07-28 (first published: 2020-07-16)
730 reads
Blogs
Giving AI Agents Visibility Into SQL Server with MCP
I’ve been thinking a lot lately about what it actually takes to make an...
The Book of Redgate: Profits
By Steve Jones
Redgate is a for-profit company. We look to make money by building and selling...
Session Materials for Techorama & DataGrillen 2026
I’ve uploaded the slides for my Techorama session Microsoft Fabric for Dummies and my...
Forums
Even When You Know What You're Doing, You Can Screw Up
Comments posted to this topic are about the item Even When You Know What...
The New Software Team
Comments posted to this topic are about the item The New Software Team
Database Mail in SQL Server 2022
Comments posted to this topic are about the item Database Mail in SQL Server...
Question of the Day
The string_agg function
We create the following table and then insert some records in it:
create table t1 ( id int primary key, category char(1) not null, product varchar(50) ); insert into t1 values (1, 'A', 'Product 1'), (2, 'A', 'Product 2'), (3, 'A', 'Product 3'), (4, 'B', 'Product 4'), (5, 'B', 'Product 5');What happens if we execute the following query in both Sql Server and PostgreSQL?
select id,
category,
string_agg(product, ';')
over (partition by category order by id
rows between unbounded preceding and unbounded following) as stragg
from t1; See possible answers