Extract, Transform, Load (ETL)

SQLServerCentral Article

The Art of Data Ingestion - Part 1

  • Article

Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]

5 (8)

You rated this post out of 5. Change rating

2022-11-11

5,253 reads

SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

  • Article

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

You rated this post out of 5. Change rating

2007-11-08

6,758 reads

Blogs

Get Configuration Change History From the SQL Server Error Logs

By

Get Configuration Change History From the SQL Server Error Logs There...

A Study in SQL Server Ad hoc Query Plans

By

A Study in SQL Server Ad hoc Query Plans ...

Read the latest Blogs

Forums

Gentlest of Reminders, Don't Stop Learning

By Grant Fritchey

Comments posted to this topic are about the item Gentlest of Reminders, Don't Stop...

table with varcahar(max)

By sqlguy80

Gurus, there is a table which is very slow to query , it has...

Query goes crazy.

By Robert-378556

Hi. I have a query that returns few rows (out of 7 table join,...

Visit the forum

Ask SSC Logo Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers