Raghavendra Narayana


SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

You rated this post out of 5. Change rating

2007-11-08

6,920 reads

Blogs

Going to Cloud? Look at the Shared Responsibility Model

By

The bottom line here is this: the idea that a CSP takes care of...

How Long Will It Take to Learn SQL? The Truth Might Surprise You

By

In today's data-driven world, SQL (Structured Query Language) is a blazing-hot skill. It's your...

Flying Halfway Around the World to Australia

By

I leave tonight for Australia. I was in London 3 days ago, so this...

Read the latest Blogs

Forums

Restore dbs

By mtz676

The below code works for backup files only from local drives. When replaced with...

Fine-tune SQL for better performance

By don075

I have been asked to fine-tune the following SQL that was written by a...

Row-level security in SQL server

By dzukik0711

In this scenario, I seek assistance in implementing row-level permissions for a table named...

Visit the forum

Question of the Day

Initial Config of tempdb

What are the initial config sizes for the tempdb primary data files, secondary data files, and log files in SQL Server 2022?

See possible answers