Blog Post

PASS 2015 Session Report – Azure SQL DW Integration with the Azure Ecosystem & End of PASS 2015

,

PASS 2015 continues (and finishes up today!) in Seattle.

Its been an amazing conference this year with a few things really hitting home;

  • Amazing technology announcements around SQL 2016 CTP3
  • Incredible advances in almost every component in Azure Data Services
  • Full and seamless SQL/Azure ecosystem integration – and by that I mean both On-Prem and/or within the Azure Cloud.  The story of either On-Prem or Azure Cloud is compelling enough individually, however the Hybrid story is now a reality for SQL and enables dynamic and flexible architectures well beyond what competitors can offer.
  • BUT what astounds me the most is actually the pace of change – barely a day goes by where I don’t receive a new services or feature update related to SQL 2016 CTP3 or Azure.
  • I don’t recall a time (in recent memory) where the step changes have come so thick/fast – its certainly changed from where I started as a DBA on RDB/VMS back in 1994 where patches arrived by mail on tape cartridge! ??
  • (As a quick aside a chief designer on RDB was Jim Gray, the same who joined Microsoft to lead the SQL Server architecture to stardom soon after Oracle bought-out and shelved DEC around 1995+)

 

Enough reminiscing already – moving along – Today I attended 5 back-back sessions, and again I cannot blog about all of them in the time I have (or want to spend), but the one which stands out the most was Azure SQL Data Warehouse and Integration with the Azure Ecosystem by Drew DiPalma of Microsoft.

This session focused specifically on the Azure ecosystem surrounding the Azure SQL Data Warehouse (SQL DW) and how it can seamlessly interact with other Azure components to create different operational solutions.  To me this was very compelling, not necessarily due to the SQL DW technology (which I know well already as the on-prem APS appliance), but more-so as it showed just how easily all parts of Azure can happily work together.

Key Takeaways

  • The Azure SQL Data Warehouse (SQL DW) is primarily composed of 2 major components; Azure SQL DB (for compute) and Azure Blob Store (for storage).
  • If you already know APS, then the easiest way to think about SQL DW is as APS in the Azure cloud, along with the same scaleable MPP architecture
  • The data size fit mentioned was from 250GB though to PB sizing.  250GB seems low to me – so I’d have to tally this up for myself to find the tipping point cost wise vs performance as to what it really is – however like anything it will be customer situation and workload dependent.
  • The compute can be scaled up or down dynamically in a few minutes (number mentioned was 2-5) depending on the need – and it can even be paused when you don’t want to do any processing (not sure when I’d pause though?  Again customer and workload dependent!).  In preview it was mentioned the compute will need to go offline to scale, but will likely be an online operation once GA.
  • As compute and storage are separately billed then “pushing the compute slider-bar” helps save you compute time and thus $$$ on your bosses credit card.
  • Pricing is calculated via the Database Warehouse Unit (DWU).  A DWU represents the power of the query performance and is quantified by workload objectives: how fast rows are scanned, loaded, and copied.
  • POLYBASE is natively built in so you can run queries across the SQL DW content within the Azure Blob Store and integrate with data outside of SQL DW, such as within your HDInight cluster, all within the very same T-SQL query plan
  • ETL/ELT/EL can be performed using POLYBASE directly (from HDInsight for example), SQL SSIS (for say on-prem SQL or SQL within Azure IaaS), or from any other Azure components such as Azure Stream Insight (for say processing event streams from Azure Event Hubs), Azure SQL DB and/or Azure Machine Learning which can connect direct to SQL DW.  And on top of all that, lets not forget Azure Data Factory (ADF) !
  • My key takeaway from this session = Just about all parts of the Azure ecosystem can be a connector for any other part and in almost any data flow direction. 
  • At the moment this is in preview – however given pace of change I’d expect GA soon!

So yet another great session!


Disclaimer: all content on Mr. Fox SQL blog is subject to the disclaimer found here 

Rate

You rated this post out of 5. Change rating

Share

Share

Rate

You rated this post out of 5. Change rating