Raghavendra Narayana


SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

6,061 reads

Blogs

What’s my MAC Address in a Command Prompt?–#SQLNewBlogger

By

Another post for me that is simple and hopefully serves as an example for...

You don’t need a blockchain

By

After writing several posts about a neat feature in Azure SQL called system-versioned ledger...

Daily Coping 27 Oct 2021

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Read the latest Blogs

Forums

Error connecting using AlwaysON Listener

By Arsh

Dear Friends, I have a 2-node Read-Scale Always ON Availability Group to which I...

How to get feature key and feature value separated $ based on part id ?

By ahmed_elbarbary.2010

I work on sql server 2012 i need to get featurekey and feature value...

Setting Up SQL Server 2019 for Optimal Performance

By water490

Hi I have a very large database (600GB) that I am looking at creating. ...

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers