Post reply

CDC - Incremental data load to DW

  • February 15, 2016 at 7:19 am

    #304671

    Hi guys,

    I would like to implement CDC on a few tables for incremental data load from OLTP db to DW.

    If any one experience on setting this up, Please share your thoughts or steps on enabling CDC and setting up SSIS for ETLs. IF anyone can provide the scripts /best practices to follow are much appreciated.

    Questions:

    =========

    How the incremental load works/setup if there are multiple updates on the same row in a certain period. Any performance impact.

    Should I include all columns for CDC tracking table/s.

    If there is a table structure change (new column added) after CDC is enabled, how does this affect the existing CDC setup?

    We do a daily refresh of the db for reporting. How this should be handled in this case. Can I run the ETL against the reporting db after refresh(restore with cdc enabled and re-enable the cdc jobs) or it should be against the OLTP db so that I can avoid ETL load against OLTP?

    How should the CDC table cleanup be setup?

    Many thanks!

  • February 15, 2016 at 7:36 am

    #1858783

    There are a lot of resources and articles already written about this (here, for example).

    Assuming that you are familiar with the tools available to you, can you be more specific with your questions?

  • February 15, 2016 at 7:56 am

    #1858787

    Thanks Phil. I went through this article before.

    Few Questions:

    How the incremental load works/setup if there are multiple updates on the same row in a certain period. Any performance impact?

    Should I include all columns for CDC tracking table/s.

    If there is a table structure change (new column added) after CDC is enabled, how does this affect the existing CDC setup?

    We do a daily refresh of the db for reporting. How this should be handled in this case. Can I run the ETL against the reporting db after refresh(restore with cdc enabled and re-enable the cdc jobs) or it should be against the OLTP db so that I can avoid ETL load against OLTP?

    How should the CDC table cleanup be setup?

Viewing 3 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply