Adding microseconds to a timestamp in Azure Data Factory

(2023-Apr-25) It’s not hard to see that many different teams were involved in building Azure Data Factory (ADF) product at Microsoft. It doesn’t have conformed constraints for naming the objects...

Using Azure Data Factory to read and process REST API datasets

(2023-Apr-10) Yes, Azure Data Factory (ADF) can be used to access and process REST API datasets by retrieving data from web-based applications.To use ADF for this purpose, you can simply...

Metadata-driven pipelines in Azure Data Factory | Part 4 - Analytical Processing

(2023-Feb-20) The previous posts covered the following areas of Metadata-driven pipelines in Azure Data Factory:Part 1 - Data CopyPart 2 - Feed ConfigurationPart 3 - Column MetadataThese 3 areas suggested that it is...

Thinking about data points and more …

(2022-Aug-28) Going on a family vacation road trip always fills me with a myriad of happy emotions and expectations to explore new places, see new people and essentially be led...

Azure DevOps: Deploying Power BI reports with a parameterized gateway-based Data Source

(2022-Aug-08) You have developed your Power BI report, by connecting one or more sourcing datasets, building your data model and enriching it with interactive and engaging visualizations. Now you are...

Azure DevOps: Merging code to Main branch from a specific branch only

(2022-July-25) I really liked reading printed PC Magazines (https://www.pcmag.com/) or Computerworld newspapers (https://www.computerworld.com/) when I was in school. Those magazines pictured IT System or Database Admins as a special group...

Including reference data into a database deployment process or passing lookup tables to Production

(2022-Jun-20) As it takes a village to raise a child, it also takes a team to develop a good SQL Server database project. Previous efforts to maintain manual or custom SQL...

Metadata-driven pipelines in Azure Data Factory | Part 3

(2022-Jun-06) This blog post is a collaborative effort of several people, some of them I currently work with, and others' contributions came from their writing, so it’s more than natural...

Populating PostgreSQL JSONB column using Azure Data Factory Data Flow

(2022-May-31) 
Personal notes of one of the recent ADF JSON pipeline development to remember how to use it next time.
Requirements:

Sourcing data comes from a SQL Server database
The destination is a...

Metadata-driven pipelines in Azure Data Factory | Part 2

(2022-May-08) If you happened to read the Part 1 of this blog series, the main message of it was the Azure Data Factory Copy activity was not just a simple...