Trigger the Execution of an ADF Pipeline from an Email
Learn how you can configure an Azure Data Factory pipeline that is triggered by an email being sent to an address.
2023-11-03 (first published: 2023-10-27)
3,582 reads
ETL/SSIS/Azure Data Factory
Top 10 Methods to Improve ETL Performance Using SSIS
Extraction Transformation Load (ETL) is the backbone for any data warehouse. In the data warehouse world data is managed by the ETL process, which consists of three processes, Extraction-Pull/Acquire data from sources, Transformation-change data in the required format and Load-push data to the destination generally into a data warehouse or a data mart.
2023-10-25
What I Wish Everyone Knew About ETL Processes - Traditional to Modern Cloud Solutions
Learn about the evolutionary journey of ETL (Extract, Transform, Load) from traditional processes to modern cloud solutions.
2023-10-18
GIT Configuration and Automated Release for Azure Data Factory
Learn a step-by-step method to save your ADF pipelines in Git and automatically release new changes to test and production.
2024-09-23 (first published: 2023-10-16)
2,223 reads
Sending Multiple Attachments in Email from Azure Data Factory Pipeline
This article shows how you can have your Azure Data Factory (ADF) pipeline send an email using a Logic App.
2023-09-29
2,654 reads
How Effective Is Your Data Load Monitoring in SQL and Python?
Learn how you can use monitor your data warehouse load process with Python code and a structured process.
2023-10-06 (first published: 2023-09-20)
1,858 reads
Unpivot Data Using Python
This article looks at manipulating data using Python to unpivot data received in a cross tab format.
2023-06-30
3,045 reads
Advantages of Kappa architecture in the Modern Data Stack
A comparison of the lambda and kappa architectures for real time ingestion and analysis of big data.
2023-06-19
4,727 reads
Python 3 Script for Generating SQL Insert Statements from CSV Data
This Python 3 script is designed to take CSV file data pasted into the csv_data variable and generate SQL insert statements that can be used to insert the data into a MySQL database. The script is easy to use and can save you a lot of time when working with large amounts of data.
2023-05-08 (first published: 2023-05-05)
4,498 reads
How to Implement CI/CD in Azure Data Factory (ADF)
Learn how you can use CI/CD with your ADF Pipelines and Azure DevOps using ARM templates.
2023-03-10
11,999 reads
Blogs
Control Flow Restartability in Azure Data Factory
I presented at SQL Saturday Pittshburgh this past weekend about populating your data warehouse...
Monday Monitor Tips: Knowing Your RPO
By Steve Jones
A customer was asking recently about the RPO for their estate, and I showed...
Webinar tomorrow: The Role of Databases in the Era of AI
By Steve Jones
I’m hosting a webinar tomorrow with Rie Merritt from Microsoft. We’ll be talking about...
Forums
best choice for index when creating a composite key
hi everyone I am planning on adding a composite key for my tables. I...
Best Approach to cutover from Test to Live with new columns and tables added
We have a Production/Live version with up-to-date data and a Test version with older...
Table Partitioning SQL server 2022
Hi All, I am currently testing the Table Partitioning to implement in SQL server...
Question of the Day
STRING_AGG's behavior
Executing the following script (Sql Server 2022), you get the table t0 with 10 rows:
CREATE TABLE t0
( id INT PRIMARY KEY
, field1 VARCHAR(1000)
, field2 VARCHAR(MAX));
INSERT INTO t0
SELECT
gs.value
, REPLICATE ('X', 1000)
, REPLICATE ('Y', 1000)
FROM generate_series(1, 10, 1) gs;
GOWhat happens if you execute the following statements?
select STRING_AGG(field1, ';') within group (order by id) from t0;
select STRING_AGG(field2, ';') within group (order by id) from t0;