Think LSNs Are Unique? Think Again - Preventing Data Loss in CDC ETL
When using only an LSN to track which changes you have processed, you can lose data. This article shows how to avoid this problem.
2025-12-15
2,481 reads
ETL (Extract Transform and Load)
A gentle introduction to dbt
Over the past years, the tool dbt – short for data build tool – has become quite popular in the data engineering world for handling such an ELT process. dbt takes on the role of the “T”, meaning it’s responsible for transforming the data in a certain data store.
2024-12-23
Time-Series Data Analysis with Apache Druid
Learn the basics of Apache Druid and how it can be used to ingest data into a data lake.
2023-11-10
14,576 reads
What I Wish Everyone Knew About ETL Processes - Traditional to Modern Cloud Solutions
Learn about the evolutionary journey of ETL (Extract, Transform, Load) from traditional processes to modern cloud solutions.
2023-10-18
Unpivot Data Using Python
This article looks at manipulating data using Python to unpivot data received in a cross tab format.
2023-06-30
3,370 reads
Advantages of Kappa architecture in the Modern Data Stack
A comparison of the lambda and kappa architectures for real time ingestion and analysis of big data.
2023-06-19
5,380 reads
Python 3 Script for Generating SQL Insert Statements from CSV Data
This Python 3 script is designed to take CSV file data pasted into the csv_data variable and generate SQL insert statements that can be used to insert the data into a MySQL database. The script is easy to use and can save you a lot of time when working with large amounts of data.
2023-05-08 (first published: 2023-05-05)
5,693 reads
The Art of Data Ingestion - Part 1
Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]
2022-11-11
6,051 reads
2021-04-23
635 reads
Chopping Off Data
An ETL issue with Excel causes issues, because the users didn't realize a limitation.
2026-01-03 (first published: 2020-10-19)
371 reads
Blogs
Runing tSQLt Tests with Claude
By Ed Elliott
Running tSQLt unit tests is great from Visual Studio but my development workflow...
Getting Your Data GenAI-Ready: The Next Stage of Data Maturity
By James Serra
I remember a meeting where a client’s CEO leaned in and asked me, “So,...
Learn Better: Pause to Review More
By Brian Kelley
If you want to learn better, pause more in your learning to intentionally review.
Forums
Execute Process Task: Issues encrypting text file using PGP in SSIS 2022
Hello SSC, Has anyone encountered this before??? I have an odd issue that I...
Azure SQL DBA certification
Hello team Can anyone share popular azure SQL DBA certification exam code? and your...
Faster Data Engineering with Python Notebooks: The Fabric Modern Data Platform
Comments posted to this topic are about the item Faster Data Engineering with Python...
Question of the Day
Which Result II
I have this code in SQL Server 2022:
CREATE SCHEMA etl;
GO
CREATE TABLE etl.product
(
ProductID INT,
ProductName VARCHAR(100)
);
GO
INSERT etl.product
VALUES
(2, 'Bee AI Wearable');
GO
CREATE TABLE dbo.product
(
ProductID INT,
ProductName VARCHAR(100)
);
GO
INSERT dbo.product
VALUES
(1, 'Spiral College-ruled Notebook');
GO
CREATE OR ALTER PROCEDURE etl.GettheProduct
AS
BEGIN
exec('SELECT ProductName FROM product;')
END;
GO
exec etl.GettheProduct
When I execute this code as a user whose default schema is dbo and has rights to the tables and proc, what is returned? See possible answers