ETL/SSIS/Azure Data Factory

External Article

What’s New in SSIS 2025?

  • Article

There’s a new version of SQL Server released and we’re mainly an on-premises SQL Server shop. We’ve been using Integration Services (SSIS) for years now for all our ETL and data integration needs. With Microsoft’s focus on cloud (Azure and Fabric), does it make sense to upgrade our SSIS packages? Are there any new features?

2026-03-27

Technical Article

Real-Time SQL Server to BigQuery Streaming ETL using CDC

  • Script

CDC Changes: The script queries the CDC tables in SQL Server to retrieve the changes (inserts, updates, deletes) since the last sync. Each change is processed with a mapped operation type (INSERT, UPDATE, DELETE).
Real-Time Streaming to BigQuery: The captured changes are streamed directly to BigQuery using its real-time insert_rows_json method, avoiding the need for batch uploads via Google Cloud Storage.
Tracking Last Sync Time: The script tracks the last synchronization time and updates it after every successful sync, ensuring no data is missed.
Low Latency: By continuously querying the CDC tables and streaming the changes, the script achieves near real-time data synchronization.

(1)

You rated this post out of 5. Change rating

2024-11-13 (first published: )

689 reads

Blogs

Funny Money: #SQLNewBlogger

By

While wandering around the documentation looking for some Question of the Day topics, I...

Why Database AI Agents need Layers?

By

Third part in my Ai series with databases. When building AI solutions within the...

Un-Migrating From the Cloud: T-SQL Tuesday #199

By

This month we have a very interesting invitation from Koen Verbeeck. He has hosted...

Read the latest Blogs

Forums

How Do the Experts Become Experts?

By Kathi Kellenberger

Comments posted to this topic are about the item How Do the Experts Become...

Stairway to Reliable Database Deployments Level 4 – Preparing for Production Deployment

By Massimo Preitano

Comments posted to this topic are about the item Stairway to Reliable Database Deployments...

Secure Communications

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Secure Communications

Visit the forum

Question of the Day

Secure Communications

As of June 2026, what is the best version of TLS to use with SQL Server?

See possible answers