Raghavendra Narayana

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

5,046 reads

Blogs

Simple-Talk Article…Understanding Azure Storage Options

By

Simple- Talk is technical journal and community hub from Redgate. This week I have the pleasure of...

T-SQL Tuesday #120 – Recap

By

The wrap up post for T-SQL Tuesday #120. 9 bloggers contributed this month. The post...

Setting Certificate Backup Permissions for an Instance–#SQLNewBlogger

By

Another post for me that is simple and hopefully serves as an example for...

Read the latest Blogs

Forums

Numeric and Decimal

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Numeric and Decimal, which is...

Union with temp table

By mcfarlandparkway

insert into dbo.details ( DetailId ,DetailName ) select distinct st.stdid ,st.name from dbo.student st...

need help with a query to avoid while loop

By mxy

i am struggling with a query to avoid while loop, please guide me if...

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers

Question of the Day

Missing Data

See possible answers