Raghavendra Narayana

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

5,029 reads

Blogs

Interview with Greg Low

By

This is the sixth interview we have done. This time our guest is Greg...

Azure SQL Database Types

By

I want to do a quick summary post of the many different types of...

Azure SQL Database – Azure Portal Updates

By

A small but useful change has been made to the Azure Portal for Data...

Read the latest Blogs

Forums

Is Data the Future of the Vibrant Web?

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Is Data the Future of...

Modifying the Dataframe in R

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Modifying the Dataframe in R

Beware, More Ransomware is Coming

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Beware, More Ransomware is Coming

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers