Designing SQL Server ETL Pipelines That Don't Break at Scale
A few lessons learned in building ETL pipelines and tricks to ensure you can easily maintain these over time.
2026-05-15
3,812 reads
A few lessons learned in building ETL pipelines and tricks to ensure you can easily maintain these over time.
2026-05-15
3,812 reads
Learn how a real time analytics solution for ETL and reporting can be built with SQL Server, Power BI, and Stream Analytics.
2026-01-26
1,464 reads
When using only an LSN to track which changes you have processed, you can lose data. This article shows how to avoid this problem.
2025-12-15
2,970 reads
Over the past years, the tool dbt – short for data build tool – has become quite popular in the data engineering world for handling such an ELT process. dbt takes on the role of the “T”, meaning it’s responsible for transforming the data in a certain data store.
2024-12-23
Learn the basics of Apache Druid and how it can be used to ingest data into a data lake.
2023-11-10
14,736 reads
Learn about the evolutionary journey of ETL (Extract, Transform, Load) from traditional processes to modern cloud solutions.
2023-10-18
This article looks at manipulating data using Python to unpivot data received in a cross tab format.
2023-06-30
3,507 reads
A comparison of the lambda and kappa architectures for real time ingestion and analysis of big data.
2023-06-19
5,809 reads
This Python 3 script is designed to take CSV file data pasted into the csv_data variable and generate SQL insert statements that can be used to insert the data into a MySQL database. The script is easy to use and can save you a lot of time when working with large amounts of data.
2023-05-08 (first published: 2023-05-05)
5,983 reads
Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]
2022-11-11
6,318 reads
By DesertDBA
I did not expect 27 responses to my T-SQL Tuesday #204 invitation, but it...
Should you adopt an off-the-shelf AI tool, partner with a specialist, build in-house, or...
By ReviewMyDB
Part 1 of my What's New in SQL Server for the Developer series. IIF,...
Talavera Office Park, Jl. TB Simatupang No.Kav. 26, RT.1/RW.1, Cilandak Bar., Kec. Cilandak, Kota...
Comments posted to this topic are about the item Switching Schema II
Comments posted to this topic are about the item A Practical Approach to Automating...
I run this query:
SELECT s.name AS schemaname , o.name AS objectname FROM sys.objects o INNER JOIN sys.schemas s ON s.schema_id = o.schema_id WHERE o.name = 'Beer'I get two rows back for the objects, named dbo.beer and etl.beer. Both the tables have identical schemas and are not partitioned. There are 25 rows in dbo.beer and 0 rows in etl.beer. I run this code:
ALTER TABLE dbo.beer switch TO etl.beerWhat happens? See possible answers