Extract, Transform, Load (ETL)

Parallel Processing of Large Volume ETL Jobs

  • Article

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

5,656 reads

Blogs

Daily Coping 21 Jan 2021

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

It’s 2021, so why are you reading this?

By

If you’re reading this, there’s a good chance you’re straight, white, male, and English-speaking....

Daily Coping 20 Jan 2021

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Read the latest Blogs

Forums

Cervical Cancer +91-9783777879 disease Plymouth

By vadic06

Cervical Cancer +91-9783777879 disease PlymouthCervical Cancer +91-9783777879 disease PlymouthCervical Cancer +91-9783777879 disease PlymouthCervical Cancer...

Germ Cell Tumor,+91-9783777879 disease Plymouth

By vadic06

Germ Cell Tumor,+91-9783777879 disease PlymouthGerm Cell Tumor,+91-9783777879 disease PlymouthGerm Cell Tumor,+91-9783777879 disease PlymouthGerm Cell...

Snapshot Folder is not deleting automatically in Transactional Replication.

By dastagiri16

Hi Team, I created transnational replication to replicate the data from one server to...

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers

Question of the Day

Amend a Commit

See possible answers