August 13, 2015 at 2:13 am
Hi,
My company Have decided to host our New Application database onto a SQL Azure Database, with the Datawarehouse being hosted on the Azure SQL Datawarehouse.
The want an incremental feed from the production database into the warehouse.
I'm against this idea as I feel they both have limited capabilities.
For example I had planned to use Change Data Capture as part of the change tracking mechanism. Is this feature available in SQL Azure.
Can anyone explain pros and cons of using these platforms.
I'm not against Azure, I would have set up a virtual network with SQL Enterprise editions.
Thanks.
August 13, 2015 at 2:19 am
SQL Database V12 does offer Change Tracking, but I am not so sure on CDC support. On the pros and cons it really does depend. The main advantage being the built in HA /DR capabilities... its pretty impressive.
August 13, 2015 at 2:30 am
One of the main disadvantages is no SQL agent.
August 13, 2015 at 2:50 am
SimonH (8/13/2015)
One of the main disadvantages is no SQL agent.
True, but I have used Azure Automation / PowerShell to schedule jobs.
August 13, 2015 at 2:56 am
Hmm maybe I need to read up some more before completely dismissing this platform.
Its the change capture that's concerning me. I have used Change Tracking but it requires a lot more work than change data capture.
I've been tasked to look into it next week.
Cheers
August 13, 2015 at 9:41 am
You also can increase or decrease the service level on demand. There's geo-duplication, all kinds of stuff. Azure SQL Database is very much a viable option for lots of databases. It has built-in backups.
But yeah, CDC isn't on the list of stuff it supports, yet.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
August 13, 2015 at 9:49 am
How would you go about migrating data on an hourly Basis into a SQL azure Data Warehouse.
The Data Factory, looks like somewhere to start, looks interesting an were going to want to
integrate with Hadoop.
I'm just looking at our current landscape, and thinking that the limitations in Azure
are going to prove too much of a headache.
August 13, 2015 at 11:36 am
SimonH (8/13/2015)
How would you go about migrating data on an hourly Basis into a SQL azure Data Warehouse.The Data Factory, looks like somewhere to start, looks interesting an were going to want to
integrate with Hadoop.
I'm just looking at our current landscape, and thinking that the limitations in Azure
are going to prove too much of a headache.
Honestly, I'm not sure. First blush, no research, set up a VM with SSIS and pump the data across. There are a lot of options for working up in Azure. It's actually more flexible, not less.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
August 13, 2015 at 11:50 am
SQL Azure should be technically up to the task. However, my understanding is that you get billed based on network transfer fees. How many GB total of data do you anticipate sitting in the cloud a year from now, and how many GB of data transfer monthly?
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
August 13, 2015 at 12:12 pm
Eric M Russell (8/13/2015)
SQL Azure should be technically up to the task. However, my understanding is that you get billed based on network transfer fees. How many GB total of data do you anticipate sitting in the cloud a year from now, and how many GB of data transfer monthly?
But isn't the suggested plan for the warehouse to be in Azure too? Then the costs are just storage, not data transfer.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
August 13, 2015 at 12:28 pm
Grant Fritchey (8/13/2015)
Eric M Russell (8/13/2015)
SQL Azure should be technically up to the task. However, my understanding is that you get billed based on network transfer fees. How many GB total of data do you anticipate sitting in the cloud a year from now, and how many GB of data transfer monthly?But isn't the suggested plan for the warehouse to be in Azure too? Then the costs are just storage, not data transfer.
When someone's 'data warehouse' is hosted in SQL Azure, is their SSIS, Informatica, Power BI, SSAS... whatever DW tools sitting in Azure too?
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
August 13, 2015 at 1:06 pm
They should be. They don't have to be. I'd sure put them up there. You'll pay for those VMs, but those costs should, by and large, be less than moving all data down to your local machines.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
August 14, 2015 at 12:55 am
Grant Fritchey (8/13/2015)
Eric M Russell (8/13/2015)
SQL Azure should be technically up to the task. However, my understanding is that you get billed based on network transfer fees. How many GB total of data do you anticipate sitting in the cloud a year from now, and how many GB of data transfer monthly?But isn't the suggested plan for the warehouse to be in Azure too? Then the costs are just storage, not data transfer.
Isn't there "compute" cost too? I only say this because I was running some intensive queries up there and noticed a nice bill a few weeks later...
August 14, 2015 at 6:44 am
Grant Fritchey (8/13/2015)
They should be. They don't have to be. I'd sure put them up there. You'll pay for those VMs, but those costs should, by and large, be less than moving all data down to your local machines.
It seems to me that for many organizations the Azure platform will only be cost effective and practical if you're "all in" in terms of your databases, remote apps, and website. Simply deciding to host your next database project in the cloud presents some logistical and economic issues in terms of where your applications, ETL, and other legacy databases reside relative to the cloud database. A one-off project with a virtical business process (maybe HR or eCommerce) will be ideal, but the more integral the cloud database is to your enterprise (like line of business, CRM, or a data warehouse), then the less beneficial it will be to have it hosted remotely with a 3rd party.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
August 14, 2015 at 8:46 am
BL0B_EATER (8/14/2015)
Grant Fritchey (8/13/2015)
Eric M Russell (8/13/2015)
SQL Azure should be technically up to the task. However, my understanding is that you get billed based on network transfer fees. How many GB total of data do you anticipate sitting in the cloud a year from now, and how many GB of data transfer monthly?But isn't the suggested plan for the warehouse to be in Azure too? Then the costs are just storage, not data transfer.
Isn't there "compute" cost too? I only say this because I was running some intensive queries up there and noticed a nice bill a few weeks later...
You pay for the service level you set. You can max that service level without additional charges. That's been my experience and the way it was explained to me.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
Viewing 15 posts - 1 through 14 (of 14 total)
You must be logged in to reply to this topic. Login to reply