Ed Elliott's Sql Developer Blog

How to test ETL Processes in production

This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...

2019-10-02

9 reads

How do test the upstream data is good in an etl process (etl testing part 3)?

In the first two parts of this series (https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ and https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/), I talked about how to unit test your business logic and integration test your ETL infrastructure code. Having...

2019-09-30

2 reads

Blogs

Reviewing 2019 Career Goals

By

According to this post, I rate 2019 as a Failure. Of the books I...

ASF 029: Tomaz Kastrun interview

By

Introduction Tomaž Kastrun is BI & DEV developer, data analyst & data science. With...

Schema Filtering in SQL Prompt

By

I love SQL Prompt, and I’m regularly impressed by the enhancements our teams continue...

Read the latest Blogs

Forums

Automatic Redaction of PII

By Steve Jones - SSC Editor

Comments posted to this topic are about the item Automatic Redaction of PII

Find Strings Containing Hex Range

By phobia42

I have a table with a varchar field.  It contains a number of characters...

How to implement row last access date column

By xr280xr

I'm creating a table to store serialized shopping cart data associated with a user...

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers