May 16, 2017 at 10:56 am
Hello Folks,
I was looking for an option to load tables(100+) from a busy OLTP instance to Azure Data Warehouse. The tables are ranging from 100K rows to 800 Million rows. Currently most of the tables have no timestamp columns in it.
Please share your thoughts on this implementation.
This is a link to CDC/SSIS implementation. I want an option with less overhead on the OLTP system.
https://docs.microsoft.com/en-us/sql/integration-services/change-data-capture/change-data-capture-ssis
Many thanks!
May 16, 2017 at 11:18 am
SQL!$@w$0ME - Tuesday, May 16, 2017 10:56 AMHello Folks,I was looking for an option to load tables(100+) from a busy OLTP instance to Azure Data Warehouse. The tables are ranging from 100K rows to 800 Million rows. Currently most of the tables have no timestamp columns in it.
Please share your thoughts on this implementation.This is a link to CDC/SSIS implementation. I want an option with less overhead on the OLTP system.
https://docs.microsoft.com/en-us/sql/integration-services/change-data-capture/change-data-capture-ssisMany thanks!
Change Tracking is a lighter touch than CDC. Take a read here, for example.
If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.
May 16, 2017 at 12:12 pm
SSIS is the slowest way to move data into Azure SQL Data Warehouse. The fastest way is using flat files and polybase to read the files into tables. It's quite a bit of programming to set this up, but you'll be happier than trying to use SSIS.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
May 16, 2017 at 12:26 pm
Phil Parkin - Tuesday, May 16, 2017 11:18 AMSQL!$@w$0ME - Tuesday, May 16, 2017 10:56 AMHello Folks,I was looking for an option to load tables(100+) from a busy OLTP instance to Azure Data Warehouse. The tables are ranging from 100K rows to 800 Million rows. Currently most of the tables have no timestamp columns in it.
Please share your thoughts on this implementation.This is a link to CDC/SSIS implementation. I want an option with less overhead on the OLTP system.
https://docs.microsoft.com/en-us/sql/integration-services/change-data-capture/change-data-capture-ssisMany thanks!
Change Tracking is a lighter touch than CDC. Take a read here, for example.
Thanks Phil!
May 16, 2017 at 12:26 pm
Grant Fritchey - Tuesday, May 16, 2017 12:12 PMSSIS is the slowest way to move data into Azure SQL Data Warehouse. The fastest way is using flat files and polybase to read the files into tables. It's quite a bit of programming to set this up, but you'll be happier than trying to use SSIS.
Thanks Grant!
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply