Extract, Transform, Load (ETL)

SQLServerCentral Article

The Art of Data Ingestion - Part 1

  • Article

Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]

5 (8)

You rated this post out of 5. Change rating

2022-11-11

5,966 reads

SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

  • Article

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

2.44 (9)

You rated this post out of 5. Change rating

2007-11-08

7,074 reads

Blogs

How To Restore Change Data Capture – CDC Tables When Restoring a SQL Server Database

By

I have recently had the privilege of working with multiple clients who have been...

KDA: Echoes of Deception - Onboarding

By

I’ll walk through the KDA UI and tackle the first challenge before diving into...

Kusto Detective Agency - Intro

By

While I mostly write about SQL Server, I have a soft spot for Kusto....

Read the latest Blogs

Forums

2022 SSIS PackageFormatVersion?

By TG

Recently I upgraded my ssis package from 2016 to 2022, but package format version...

officialsprinklezshop.com/Buy Sprinklez Cotton Candy Vanilla Swirl Strain

By favouritechemist

Website: https://officialsprinklezshop.com/ Buy Best quality Sprinklez Strain from Officail Sprinklez Shop Sprinklez Cotton Candy...

https://officialsprinklezshop.com/ Sprinklez Super Sweet Sugar Sparks Strain

By favouritechemist

Website: ttps://officialsprinklezshop.com/ Buy Top Notch quality product from Official Sprinklez Shop. 100% Trusted -...

Visit the forum

Question of the Day

The Monitor Server

What does the monitor server do in log shipping?

See possible answers