Incremental Data loading through ADF using Change Tracking

Question

Incremental Data loading through ADF using Change Tracking

Sucharita Das

SSC-Addicted

Points: 456
More actions
October 13, 2020 at 12:00 am

#3793268

Comments posted to this topic are about the item Incremental Data loading through ADF using Change Tracking

Viewing 6 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic. Login to reply

gauri Newbie Points: 2 More actions · Answer 1

Hello Sucharita Das,

I went through your blog which talks about incremental data load with Change Tracking. In the blog its only implemented for one table. I am trying to modify the query for multiple tables but facing some issues, how can we modify this to implement for multiple tables.

Thanks,

Gauri Pawse

Sucharita Das SSC-Addicted Points: 456 More actions · Answer 2

Some minor changes are required to handle multiple tables. I will try to write an article on this.

shamul Newbie Points: 2 More actions · Answer 3

Hi Sucharita Das,

It will be good to understand how the same pipeline can be configured to handle incremental load for multiple tables using change tracking.

Thanks!

prapul Newbie Points: 1 More actions · Answer 4

prapul

Newbie

Points: 1

August 20, 2021 at 6:04 am

#3921990

Can you explain for Multiple tables

onuromer Newbie Points: 1 More actions · Answer 5

Hi,

As far as I've understood, this solution requires enabling change tracking in SOURCE Database. Right?

If so, unfortunately it is in most scenarios not applicable and therefore this pattern will be useless. In serious environments one can just forget modifying ERP (or any other heavy OLTP Datasource) tables and set the change tracking on. In such cases, the classical anchor based incremental loading pattern is still the best way.

Thanks for the information anyway.

Onur

Ps: For whom wants to learn more on Change Tracking, here are also two good articles from Tim Mitchell

- https://www.timmitchell.net/post/2016/01/18/getting-started-with-change-tracking-in-sql-server/

- https://www.timmitchell.net/post/2016/01/20/using-sql-server-change-tracking-for-incremental-loads/