SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

External Article

SQL Puzzle 7

After a gap of sixteen months, Lionel Clarke, the creator of the only 3-D maze game to be written in TSQL, returns with a new SQL puzzle. It is very simple. You have to move as much of the data as you can from the source tables to the destination tables.

Blogs

Friday Basics: the CIA Triad

By

In information security (INFOSEC), there several foundational concepts and principles. One of the ones...

A New Word: the standard blues

By

the standard blues– n. the dispiriting awareness that the twists and turns of your...

How Redgate Flyway Can Boost Your DevOps Journey

By

A brief introduction to the tool and its advantages for database migrations DevOps is...

Read the latest Blogs

Forums

One more reason to use foreign key constraints

By Louis Davidson (@drsql)

Comments posted to this topic are about the item One more reason to use...

client_app_name is empty in Extended Events output but present in sp_who2

By Pete Bishop

I'm tracing activity on one database and would like to include the client_app_name in...

How to compare data in customer table with other customers to find related cust

By Zond Sita

select Custno, Addr1, City, Res_Phone, Bus_Phone, Fax_Phone, Marine_Phone, Pager_Phone, Other_Phone, email1, email2 from customer...

Visit the forum

Question of the Day

The Marked Transaction

I want to mark a transaction in the log as a recovery point. How do I do this in my code if I use the transaction, myTran?

See possible answers