Raghavendra Narayana


SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

6,266 reads

Blogs

Daily Coping 20 May 2020

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Daily Coping 19 May 2022

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Running dynamic SQL in a different database

By

This is one of those things that on hindsight was a stupid problem, but...

Read the latest Blogs

Forums

Running Trace - Error: Login timeout expired

By ramana3327

Hi All, We have Project server 2019 (Sharepoint). The database server (Only Database services...

query perf issue

By siri_28

Hi Everyone, Good morning. I am new to Sql administration.Have one question. We work...

Looking for some guidance on avoiding Scalar UDFs for DB lookups

By SteveOC

Hello. I am not a SQL Developer - just an IT generalist (if that)...

Visit the forum

Ask SSC Logo Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers