Scripts

Technical Article

Real-Time SQL Server to BigQuery Streaming ETL using CDC

CDC Changes: The script queries the CDC tables in SQL Server to retrieve the changes (inserts, updates, deletes) since the last sync. Each change is processed with a mapped operation type (INSERT, UPDATE, DELETE).
Real-Time Streaming to BigQuery: The captured changes are streamed directly to BigQuery using its real-time insert_rows_json method, avoiding the need for batch uploads via Google Cloud Storage.
Tracking Last Sync Time: The script tracks the last synchronization time and updates it after every successful sync, ensuring no data is missed.
Low Latency: By continuously querying the CDC tables and streaming the changes, the script achieves near real-time data synchronization.

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

(1)

You rated this post out of 5. Change rating

2024-11-13 (first published: )

750 reads

Technical Article

Concatenating Multiple Row Values into a Single Comma-Separated List

In scenarios where you need to consolidate multiple rows into a single, comma-separated value, you can achieve this using FOR XML PATH. This script demonstrates how to retrieve volunteer data and display the days they have selected for participation.

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

You rated this post out of 5. Change rating

2024-11-11 (first published: )

1,822 reads

Blogs

Goodbye, Microsoft

By

A few years ago I took a new job at Microsoft, working as a...

The SUM of Nothing: #SQLNewBlogger

By

I caught this interesting item over on Pinal Dave’s blog: Eleven Interview Questions that...

How to Move the SSMS Status Bar to the Top and Color-Code SQL Server Connections

By

I use color-coded connections in SSMS to distinguish Production, Pre-Production, UAT, and Development. But...

Read the latest Blogs

Forums

Today's AI

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Today's AI

Split Large Queries in Athena with a Simple Modulo Trick

By Rahul Gupta

Comments posted to this topic are about the item Split Large Queries in Athena...

Finding Trailing Spaces

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Finding Trailing Spaces

Visit the forum

Question of the Day

Finding Trailing Spaces

I have some data in a SQL Server 2025 database. It looks like this for the dbo.Customer table:

CustomerID CustomerName PreferredName
1          Steve        Steve               
2          Andy         Andy               
3          Brian        Brian               
4          Allan        Allan               
5          Devin        Devin               
6          Steve        Steve               
7          Sally        Sally
I want to detect which names have a single trailing space. The CustomerName is a varchar() and the PreferredName is a CHAR(). Does this query detect the problem rows?
SELECT 
       CustomerID,
       CustomerName,
       PreferredName
FROM Customer
WHERE CustomerName <> RTRIM(CustomerName)
OR PreferredName <> RTRIM(PreferredName);

See possible answers