The Ever Expanding Data Platform

Question

The Ever Expanding Data Platform

Steve Jones - SSC Editor

SSC Guru

Points: 734416
More actions
September 29, 2018 at 11:11 am

#379674

Comments posted to this topic are about the item The Ever Expanding Data Platform

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply

netmikem SSCarpal Tunnel Points: 4001 More actions · Answer 1

Really enjoyed this editorial Steve so thanks for that.
I've only dabbled in Azure thus far but I think it's high time that I dive into it, starting with Azure Data Studio.
The breadth of the platform is certainly much more than what it used to be!
P.S. Good luck to your Rockies in the post season.

xsevensinzx One Orange Chip Points: 25560 More actions · Answer 2

I for sure am loving the Azure platform. I've obviously mentioned it many times over in other threads, but just to emphasize some more, I've fully migrated to Azure Data Lake Store, Azure Data Lake Analytics and completely replaced my existing on-prem SQL Server with Azure Data Warehouse. All of this is working together and it's been a good experience. Now, I'm starting to branch out from keeping everything in the data warehouse and moving subsets of data to Azure DB.

The biggest surprises for me as someone coming from just relying on SQL Server so much is how powerful the data lake store and analytics products are. Loading a TB of data and being able to run a U-SQL query on the raw CSV files that JOIN and aggregate the data without indexes, statistics or anything is insanely good.

The biggest troubles for me with this new set of tools in this new data platform is trying to navigate all the "gotchas" with working with new tech. For example, string splitters and even UPDATE JOIN's were difficult in Azure Data Warehouse, encoding issues were issues with the data lake store, not reading the data more than you need and anything that uses the .NET libraries as a non-dot net developer was difficult with analytics.

But all-in-all, pretty excited for all the new stuff with Azure that has been announced or planned.

Steve Jones - SSC Editor SSC Guru Points: 734416 More actions · Answer 3

netmikem - Sunday, September 30, 2018 5:43 PM
Really enjoyed this editorial Steve so thanks for that.
I've only dabbled in Azure thus far but I think it's high time that I dive into it, starting with Azure Data Studio.
The breadth of the platform is certainly much more than what it used to be!
P.S. Good luck to your Rockies in the post season.

Thanks, fingers crossed they make a win tonight and take the first division title and then at least have a good series. On Sept 1, they had like a 13% chance of making the playoffs, so I'm excited.

I might also recommend this book as an overview. It's free from MS: https://azure.microsoft.com/en-us/campaigns/developer-guide/?v=18.31
It's an overview of lots of their services. Easy to read, and gives you a good look at choices. It's still mostly relevant after last week's Ignite announcements.

Azure Data Studio, worth trying. Can't decide what I think, but welcome to see more thoughts from others.

Steve Jones - SSC Editor SSC Guru Points: 734416 More actions · Answer 4

xsevensinzx - Sunday, September 30, 2018 8:02 PM
I for sure am loving the Azure platform. I've obviously mentioned it many times over in other threads, but just to emphasize some more, I've fully migrated to Azure Data Lake Store, Azure Data Lake Analytics and completely replaced my existing on-prem SQL Server with Azure Data Warehouse. All of this is working together and it's been a good experience. Now, I'm starting to branch out from keeping everything in the data warehouse and moving subsets of data to Azure DB.
The biggest surprises for me as someone coming from just relying on SQL Server so much is how powerful the data lake store and analytics products are. Loading a TB of data and being able to run a U-SQL query on the raw CSV files that JOIN and aggregate the data without indexes, statistics or anything is insanely good.
The biggest troubles for me with this new set of tools in this new data platform is trying to navigate all the "gotchas" with working with new tech. For example, string splitters and even UPDATE JOIN's were difficult in Azure Data Warehouse, encoding issues were issues with the data lake store, not reading the data more than you need and anything that uses the .NET libraries as a non-dot net developer was difficult with analytics.
But all-in-all, pretty excited for all the new stuff with Azure that has been announced or planned.

I'd love some articles ;), I think there is some amazing stuff in there, and despite the potential performance issues with Polybase, I think this is great technologies in spots, and with some development to limit ad hoc querying that might result in a poor experience for the end user.