Raghavendra Narayana


Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

5,963 reads

Blogs

Data Modeling at your fingertips

By

(2021-July-30) A picture is worth a thousand words, the same way a visual database schema...

How to put a SQL Server Database into Recovery Pending

By

You might be asking why on earth would you want to get a database...

Daily Coping 30 Jul 2021

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Read the latest Blogs

Forums

Migrate SQL Sever 2012 Database To Server 2014 and Subsequent Versions

By VSCurtis

Hi Guys, I have an SQL Server 2012 Express database that I would like...

Deploying SQL Server Automatically

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Deploying SQL Server Automatically

Setting data compression on a database or even server level possible?

By DiStas

I well realize data compression is a subject to a few variables, such as...

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers