diponkar.paul

Diponkar Paul has 12 years of experiences in IT industry and specialized in Business Intelligence domain, currently working as a Senior Data Engineer at Fiera Capital, Canada. Through his career, he designs and developed medium and large data warehouses and delivered business critical analytical solution. He was leading Toronto PASS Group; which is part of the world largest data professional community named PASS and Lead of Toronto Data Professionals Community.He got diverse background, has been working and studied in different Geo location includes Canada, Denmark, UK, Sweden and Bangladesh. He writes technical blog and speaking, he loves to share his learning within the community.
  • Interests: Data, Travel, Swimming, Cycling, cross-country skate
  • Blog: http://allaboutdata.ca/
  • Jobs: Senior Data Engineer
  • Skills: Big data, Business Intelligence, Database, ETL, ELT, Data Warehouse

SQLServerCentral Article

How to Recover a Corrupted Azure Data Factory Integration Runtime

I would like to share my recent experience with Azure Data Factory (ADF) where AutoResolveIntegrationRuntime become corrupted and how did I recover it. I still don't know how the Integration Runtime (IR) was corrupted. However, if it happens, then this article will help you to solve the issue. Problem In general, the ADF AutoResolveIntegrationRuntime should […]

5 (1)

2021-11-05

638 reads

SQLServerCentral Article

How to Flatten JSON in Azure Data Factory?

When you work with ETL and the source file is JSON, many documents may get nested attributes in the JSON file. Your requirements will often dictate that you flatten those nested attributes. There are many ways you can flatten the JSON hierarchy, however; I am going to share my experiences with Azure Data Factory (ADF) […]

4.5 (2)

2021-09-17

3,164 reads

SQLServerCentral Article

Handling Row-level Errors in ADF Data Flows

If you are working with ADF (Azure Data Factory) data flows, then you may have noticed there was a new feature released in November 2020, which is useful to capture any error while inserting/updating the records in a SQL database. This article will describe how to setup the error row handling feature and why it's […]

5 (1)

2021-06-16 (first published: )

1,832 reads

SQLServerCentral Article

Step by Step Guide to Install Jupyter Notebook

Whether you work as a Data Engineer or a Data Scientist, a Jupyter Notebook is a helpful tool. One of the projects I was working required a comparison of two parquet files. This is mainly a schema comparison, not a data comparison. Though the two .parquet were created from two different sources, the outcome should […]

2021-05-17

3,380 reads

SQLServerCentral Article

Dynamically Add a Timestamp To Files in Azure Data Factory

This article will describe how to add your local timestamp at the end of the each file in Azure Data Factory (ADF). In general, ADF gets a UTC timestamp, so we need to convert the timestamp from UTC to EST, since our local time zone is EST. For example, if the input Source file name […]

5 (2)

2021-04-22

7,461 reads

SQLServerCentral Article

How to simulate Case statement in Azure Data Factory (ADF) compared with SSIS?

This post will describe how you use a CASE statement in Azure Data Factory (ADF). If you are coming from SSIS background, you know a piece of SQL statement will do the task. However let's see how do it in SSIS and the very same thing can be achieved in ADF. Problem statement For my […]

4.5 (2)

2020-11-12

7,384 reads

Blogs

Daily Coping 6 Dec 2021

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Dec 8 Webinar with Pavilion – Boost SQL Server on VMware Performance 400% with 2-Step Tuning

By

Join us on December 8th at 10am Pacific time for an exciting new webinar!...

Daily Coping 3 Dec 2021

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Read the latest Blogs

Forums

Problem understanding SUM function with GROUP BY

By DaveBriCam

I have code I'm trying to write to sum the time in decimal for...

Problem understanding SUM function with GROUP BY

By DaveBriCam

I have code I'm trying to write to sum the time in decimal for...

List file shares from TAR file

By Bruin

How can I list just the directories from a TAR file, then extract just...

Visit the forum

Ask SSC Logo Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers