Revolutionizing Real-Time Data Management with SQL Server, Kafka, and Informatica
This article examines how one can structure a pipeline for processing real-time data using Kafka and Informatica.
2023-04-26
3,633 reads
data engineering
Step by Step Guide to Install Jupyter Notebook
Whether you work as a Data Engineer or a Data Scientist, a Jupyter Notebook is a helpful tool. One of the projects I was working required a comparison of two parquet files. This is mainly a schema comparison, not a data comparison. Though the two .parquet were created from two different sources, the outcome should […]
2021-05-17
4,316 reads
Dynamically Add a Timestamp To Files in Azure Data Factory
This article will describe how to add your local timestamp at the end of the each file in Azure Data Factory (ADF). In general, ADF gets a UTC timestamp, so we need to convert the timestamp from UTC to EST, since our local time zone is EST. For example, if the input Source file name […]
2021-04-22
21,958 reads
Google Cloud Certification for Microsoft Database Engineer
Introduction I recently passed the Google Cloud Professional Data Engineer certification exam, Professional Data Engineer Certification. It took me about five month to prepare for this, and I would like to share my thoughts of why I decided to take it on and how I prepared for it. At the moment, Google cloud (GCP) is […]
2021-03-15
9,425 reads
Blogs
Microsoft Announcement: Microsoft Fabric
Yesterday at Microsoft Build, a significant announcement took place—the introduction of Microsoft Fabric, which...
Introduction to SQL Server Query Store
Introduced with SQL 2016, Query Store was, probably without doubt, the most anticipated and...
Flyway with Multiple Folders
By Steve Jones
A customer asked about how they could organize their migration scripts in different ways...
Forums
You have an error in your SQL syntax; check the manual that corresponds
I am having great trouble creating a data pipeline using MariaDB connector 3 with...
Availability Group database went into 'Restoring' state
Dear Friends, Please advise why would one of my Availability Group databases go into...
Access db
Access db wanting to move sql server, how much control does business have over...
Ask SSC