Raghavendra Narayana

All authors

SQLServerCentral Article

Parallel Processing of Large Volume ETL Jobs

ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

(9)

You rated this post out of 5. Change rating

2007-11-08

7,103 reads

All authors

Blogs

In-Person CISA Training – April 13-16, 2026

By

I will be leading an in-person Certified Information Systems Auditor (CISA) exam prep class...

EightKB 2026

By

EightKB is back again for 2026! The biggest online SQL Server internals conference is...

The FinOps Lifecycle: From Budgeting to Reporting

By

Working in DevOps long enough teaches you two universal truths: That’s exactly why I...

Read the latest Blogs

Forums

VS Code, Unresolved References.

By mjdemaris

Hi all, I just started using VS Code to work with DB projects.  I...

Fun with JSON II

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Fun with JSON II

Changing Data Types

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Changing Data Types

Visit the forum

Question of the Day

Fun with JSON II

I have some data in a table: 

CREATE TABLE #test_data
(
    id INT PRIMARY KEY,
    name VARCHAR(100),
    birth_date DATE
);

-- Step 2: Insert rows  
INSERT INTO #test_data
VALUES
(1, 'Olivia', '2025-01-05'),
(2, 'Emma', '2025-03-02'),
(3, 'Liam', '2025-11-15'),
(4, 'Noah', '2025-12-22');
If I run this query, how many rows are returned? 
SELECT t1.[key] AS row,
       t2.*
FROM OPENJSON(
     (
         SELECT t.* FROM #test_data AS t FOR JSON PATH
     )
             ) t1
    CROSS APPLY OPENJSON(t1.value) t2;

See possible answers