SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

External Article

SQL Puzzle 7

After a gap of sixteen months, Lionel Clarke, the creator of the only 3-D maze game to be written in TSQL, returns with a new SQL puzzle. It is very simple. You have to move as much of the data as you can from the source tables to the destination tables.

Blogs

SQL Server Journey Part 2: Modern Era (2017 – 2026) – AI/Cloud First

By

Following up on my Part 1 baseline, the journey from 2017 onward changed how...

Google Moves Up Post-Quantum Cryptography Timeline

By

In cryptography, the RSA and ECC algorithms which we use primarily for asymmetric cryptography...

The Book of Redgate: No Politics

By

In today’s world, this might mean something different, but in 2010, we had this...

Read the latest Blogs

Forums

Dancing Robot Goes Rogue

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Dancing Robot Goes Rogue

advice as i migrate my winscp based ssis pkg to our prod server

By stan

Hi , i installed winscp on my pc, added it to GAC thru vs...

Identities and Sequences II

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Identities and Sequences II

Visit the forum

Question of the Day

Identities and Sequences II

In thinking about the differences between the identity property and a sequence object, which of these two guarantees that there are consecutive numbers (according to the increment) inserted in a single table?

See possible answers