Raghavendra Narayana


Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

5,589 reads

Blogs

Goal Progress–November 2020

By

This is my report, which continues on from the Oct report. It’s getting near...

Azure Synapse Analytics is GA!

By

(Note: I will give a demo on Azure Synapse Analytics this Saturday Dec 5th...

5 Things You Should Know About Azure SQL

By

Azure SQL offers up a world of benefits that can be captured by consumers...

Read the latest Blogs

Forums

Database, RAM and processor Information

By sizal0234

Hi, I need to find the size of all db's and server's processor and...

reference for writing stored procedures (especially spatial) in DB2

By pietlinden

I can write stored procedures in SQL Server ... is there a guide somewhere...

A Technology Collapse

By Steve Jones - SSC Editor

Comments posted to this topic are about the item A Technology Collapse

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers