Raghavendra Narayana


SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

You rated this post out of 5. Change rating

2007-11-08

7,083 reads

Blogs

The PASS Summit on Tour in Dallas

By

The PASS Summit goes on tour this year, with a September stop in Dallas....

SQL Server Auto-settings: The Good, the Bad, and the Ugly

By

Settings That Could Be Hurting Your Performance If you’ve ever created a new SQL...

Bad Request Error when Running PowerShell Command in Azure Devops

By

I needed to run a PowerShell cmdlet in an Azure Devops pipeline. The cmdlet...

Read the latest Blogs

Forums

How a Legacy Logic Choked SQL Server in a 30-Year-Old Factory

By Chandan Shukla

Comments posted to this topic are about the item How a Legacy Logic Choked...

Has the PK been used --

By dbakevlar

Comments posted to this topic are about the item Has the PK been used...

Has the PK been used --

By dbakevlar

Comments posted to this topic are about the item Has the PK been used...

Visit the forum

Question of the Day

Has the PK been used --

You're analyzing a query execution plan in SQL Server to determine whether the query optimizer chose to use a primary key index. Which Dynamic Management View (DMV) would you use to verify the usage of the primary key in the plan?

See possible answers