Extract, Transform, Load (ETL)

SQLServerCentral Article

The Art of Data Ingestion - Part 1

  • Article

Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]

5 (8)

You rated this post out of 5. Change rating

2022-11-11

6,025 reads

SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

  • Article

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

You rated this post out of 5. Change rating

2007-11-08

7,089 reads

Blogs

Leadership in Times of Change: Guiding Teams Through Uncertainty, Disruption, and Transformation

By

Change is inevitable. What separates thriving organizations from those that falter is not the...

The End of Year Vacation Burn

By

It’s been a good year for me. With my sabbatical, I felt like I’ve...

Speaking at PASS Summit 2025

By

In less than one month, I will be speaking not once but twice at...

Read the latest Blogs

Forums

Create an HTML Report on the Status of SQL Server Agent Jobs

By Nisarg Upadhyay

Comments posted to this topic are about the item Create an HTML Report on...

how to write this query?

By water490

Hi everyone I am a bit stuck on how to write this query. Below...

Powershell script to generate SQL server Inventory in CSV format

By Srinivas Merugu

Comments posted to this topic are about the item Powershell script to generate SQL...

Visit the forum

Question of the Day

Azure SQL Database MAXDOP

When Azure SQL Database was originally released, the MAXDOP was set to 0 for each new database. In Oct 2025, what is the default value for Azure SQL databases?

See possible answers