Revolutionizing Real-Time Data Management with SQL Server, Kafka, and Informatica
This article examines how one can structure a pipeline for processing real-time data using Kafka and Informatica.
2023-04-26
3,633 reads
This article examines how one can structure a pipeline for processing real-time data using Kafka and Informatica.
2023-04-26
3,633 reads
Whether you work as a Data Engineer or a Data Scientist, a Jupyter Notebook is a helpful tool. One of the projects I was working required a comparison of two parquet files. This is mainly a schema comparison, not a data comparison. Though the two .parquet were created from two different sources, the outcome should […]
2021-05-17
4,316 reads
This article will describe how to add your local timestamp at the end of the each file in Azure Data Factory (ADF). In general, ADF gets a UTC timestamp, so we need to convert the timestamp from UTC to EST, since our local time zone is EST. For example, if the input Source file name […]
2021-04-22
21,958 reads
Introduction I recently passed the Google Cloud Professional Data Engineer certification exam, Professional Data Engineer Certification. It took me about five month to prepare for this, and I would like to share my thoughts of why I decided to take it on and how I prepared for it. At the moment, Google cloud (GCP) is […]
2021-03-15
9,425 reads
Yesterday at Microsoft Build, a significant announcement took place—the introduction of Microsoft Fabric, which...
Introduced with SQL 2016, Query Store was, probably without doubt, the most anticipated and...
By Steve Jones
A customer asked about how they could organize their migration scripts in different ways...
I am having great trouble creating a data pipeline using MariaDB connector 3 with...
Dear Friends, Please advise why would one of my Availability Group databases go into...
Access db wanting to move sql server, how much control does business have over...