Extract, Transform, Load (ETL)

SQLServerCentral Article

The Art of Data Ingestion - Part 1

  • Article

Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]

5 (8)

2022-11-11

4,669 reads

SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

  • Article

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

2007-11-08

6,582 reads

Blogs

FIRST Robotics Competition 2023 - World Championships!

By

On to the FIRST Robotics Competition 2023 World Championship Last week I wrote about...

Errata for my book: Pro Encryption in SQL Server 2022

By

My biggest fear when my book went into production was that any factual errors...

SQL Server Quickie #45 – SQL Server Availability Groups

By

Today I have uploaded SQL Server Quickie #45 to YouTube. This time I’m talking...

Read the latest Blogs

Forums

Query based on sorting based on month

By Smash125

    SELECT BusinessEntityID,FirstName,MiddleName,LastName, YEAR(OrderDate) AS OrderYear, DATENAME(MM,OrderDate) AS OrderMonth FROM Person.Person Pp INNER...

SSRS 2017 - SSRS 2019 migration

By Wecks

I am after some advice, I have completed a SSRS migration 2017 - 2019. ...

Fixing a mismatched collation on MSDB

By NoDowt

Hi, I've been tasked with finding out why we've been unable to patch the...

Visit the forum

Ask SSC Logo Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers

Question of the Day

Cloud Diagrams

See possible answers