March 24, 2018 at 1:51 pm
Comments posted to this topic are about the item So Many Choices with SQL Server
Aunt Kathi Data Platform MVP
Author of Expert T-SQL Window Functions
Simple-Talk Editor
March 25, 2018 at 1:07 am
I must be missing something very important with all the "cloud" (remote hosting) options.
If I'm over here (such as in an office) and my data is over there (such as at a remote hosting facility), and the only path between here and there is the Internet, and I need to work with a large quantity of my data, how is it even possible to do that efficiently? How do I avoid dragging a large quantity of my data over the Internet, which is going to be much slower than a typical LAN? What if I need to create an exact copy of the production database for tracking down a problem? What if I need to populate a development database with production data? What if I need to create a large data extract for a business partner? What if I need to have a large quantity of data locally for analysis by business people using various tools? What if I need to have a large quantity of data locally for application testing or some kind of research?
I have run into all of these needs many times in my career. It's not clear to me how it's at all practical to handle these needs if/when the production database is remote.
My current job is 100% remote. I sometimes run queries that return quite a bit of data. If I run such a query from a machine within the corporate network the performance might be fine. If I run such a query from a machine in my home office the performance might be awful. The difference is dragging the data over the corporate network versus dragging the data over the Internet and into my house. It's not like I have a poor Internet connection. I just did a speed test on my Internet connection and I had better then 140 Mbps performance.
A previous employer of mine had an ERP system hosted elsewhere. Their business transactions were fine, but many other data operations were severely hampered by performance issues. They could have obtained a faster connection to the hosting facility, but it was extremely expensive.
How are people handling common data tasks, such as those I have mentioned here, when they are using remote hosting for their database(s)?
As more businesses use remote hosting for their database(s), at what point will the additional traffic over the Internet become an issue?
Many companies do not allow audio/video streaming over their corporate networks because it chews up bandwidth on their Internet connection. How is it not a similar issue if people are constantly accessing a remote database for operational reporting, analytic reporting, or other data intensive tasks?
Creator of SQLFacts, a free suite of tools for SQL Server database professionals.
March 25, 2018 at 3:21 pm
Wingenious - Sunday, March 25, 2018 1:07 AMIf I'm over here (such as in an office) and my data is over there (such as at a remote hosting facility), and the only path between here and there is the Internet, and I need to work with a large quantity of my data, how is it even possible to do that efficiently? How do I avoid dragging a large quantity of my data over the Internet, which is going to be much slower than a typical LAN? What if I need to create an exact copy of the production database for tracking down a problem? What if I need to populate a development database with production data? What if I need to create a large data extract for a business partner? What if I need to have a large quantity of data locally for analysis by business people using various tools? What if I need to have a large quantity of data locally for application testing or some kind of research?
I've done a ton of work for clients remotely and I never pull their data to my laptop. I always remote in to their systems. So, maybe that's not quite the same as having the database in Azure. I'm interested in hearing how others handle this.
Aunt Kathi Data Platform MVP
Author of Expert T-SQL Window Functions
Simple-Talk Editor
March 26, 2018 at 4:24 am
@wingenious we try and manage with much less access essentially.
I guess to an extent that's planning what you're doing accordingly. Maybe (for our use case, yours would no doubt be different) using Stackify's Retrace on stage servers to ensure serious performance issues don't get to production servers. Data extracts can be done with queries or Data Compare to only take what is necessary. Additionally we aren't able to do full copies of production in many cases with current regulations, let alone GDPR looming, but we might do an anonymised copy to dev overnight. It does mean working a bit differently to having everything available on-site, but we have managed the transition fairly successfully - it's much harder for some than others of course.
March 27, 2018 at 7:49 am
If you are concerned about how much data would be pulled across the network from Azure to your work PC, then one option is to provision an Azure IaaS instance for development and analytical use. It's basically an Azure hosted VM upon which you remote desktop into and install SSMS, Visual Studio and whatever other tools you need to work with.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply