Extract, Transform, Load (ETL)

Parallel Processing of Large Volume ETL Jobs

  • Article

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

5,108 reads

Blogs

Power BI Monthly Digest – February 2020

By

In this month’s Power BI Digest Manuel I will again guide you through some...

Using Google Chrome with SSRS

By

In this post, you will learn how to resolve the Kerberos Double-Hop issue and...

Using Google Chrome with SSRS

By

In this post, you will learn how to resolve the Kerberos Double-Hop issue and...

Read the latest Blogs

Forums

More Data

By Steve Jones - SSC Editor

Comments posted to this topic are about the item More Data

SQL 2019 AG and system databases

By tony28

Hello, I read that in new SQL is Ag feature also for sync system...

Dimensional Database \"examples\"

By wallywizard

I have read many things online regarding dimensional vs relational databases. I under that...

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers

Question of the Day

More Data

See possible answers