How Effective Is Your Data Load Monitoring in SQL and Python?

Learn how you can use monitor your data warehouse load process with Python code and a structured process.

Unpivot Data Using Python

This article looks at manipulating data using Python to unpivot data received in a cross tab format.

Advantages of Kappa architecture in the Modern Data Stack

A comparison of the lambda and kappa architectures for real time ingestion and analysis of big data.

Python 3 Script for Generating SQL Insert Statements from CSV Data

This Python 3 script is designed to take CSV file data pasted into the csv_data variable and generate SQL insert statements that can be used to insert the data into a MySQL database. The script is easy to use and can save you a lot of time when working with large amounts of data.

How to Implement CI/CD in Azure Data Factory (ADF)

Learn how you can use CI/CD with your ADF Pipelines and Azure DevOps using ARM templates.

Ingesting Data from Google Drive to Azure Data Lake Gen2 using Azure Data Factory

Learn how to setup, configure and ingest data from Google Drive using Azure Data Factory in this step-by-step article.

The Art of Data Ingestion - Part 1

Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]

The Assert Transformation

Migrate the SSIS Catalog in Just a Few Clicks

This wizard migrates SSIS Catalog from one server to another in just a few clicks.

Assert Transformations in Azure Data Factory

This article will show how you can use Azure Data Factory to check your data quality with an assertion.