SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Azure DWH part 14: PolyBase to access to non structured data


Azure DWH part 14: PolyBase to access to non structured data

Author
Message
calbimonte.daniel
calbimonte.daniel
SSCarpal Tunnel
SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)

Group: General Forum Members
Points: 4261 Visits: 528
Comments posted to this topic are about the item Azure DWH part 14: PolyBase to access to non structured data
xsevensinzx
xsevensinzx
SSC-Insane
SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)

Group: General Forum Members
Points: 23028 Visits: 6101
What happens if the structure of the CSV file changes? Does it break the External Table? What happens if two files with two different structures exist in the same directory location?

How does this article help me with unstructured data?
calbimonte.daniel
calbimonte.daniel
SSCarpal Tunnel
SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)

Group: General Forum Members
Points: 4261 Visits: 528
You will have errors if the structure change. The same with files with different format.
This article will help with non relational data. We will update the tittle.
Incredible Mouse
Incredible Mouse
Right there with Babe
Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)

Group: General Forum Members
Points: 756 Visits: 251
My world is so different, in that all of our data must stay on premise, that all these articles that spend 99% of the space explaining azure wastes so much of my time. :{ I may be alone in this, or maybe not?
calbimonte.daniel
calbimonte.daniel
SSCarpal Tunnel
SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)SSCarpal Tunnel (4.3K reputation)

Group: General Forum Members
Points: 4261 Visits: 528
Last year there were 120K new Azure Subscribers per month, 1.4 million of Azure SQL Databases, 5 million organizations using Azure AD, 4 million developers register with Visual Studio.
The Microsoft Share Revenue is growing because of Azure.
Incredible Mouse
Incredible Mouse
Right there with Babe
Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)Right there with Babe (756 reputation)

Group: General Forum Members
Points: 756 Visits: 251
calbimonte.daniel - Monday, August 21, 2017 8:25 AM
Last year there were 120K new Azure Subscribers per month, 1.4 million of Azure SQL Databases, 5 million organizations using Azure AD, 4 million developers register with Visual Studio.
The Microsoft Share Revenue is growing because of Azure.

More impressive than I would have guessed. How many developers/analysts have a local or local network access to a SQL Server 2016+ instance, that could bypass Azure?

xsevensinzx
xsevensinzx
SSC-Insane
SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)SSC-Insane (23K reputation)

Group: General Forum Members
Points: 23028 Visits: 6101
INCREDIBLEmouse - Monday, August 21, 2017 9:21 AM
calbimonte.daniel - Monday, August 21, 2017 8:25 AM
Last year there were 120K new Azure Subscribers per month, 1.4 million of Azure SQL Databases, 5 million organizations using Azure AD, 4 million developers register with Visual Studio.
The Microsoft Share Revenue is growing because of Azure.

More impressive than I would have guessed. How many developers/analysts have a local or local network access to a SQL Server 2016+ instance, that could bypass Azure?

I don't in my example. But that's not why I am using Azure (assuming you're referring to needing Polybase). I particularly use Azure for Data Warehousing and Data Science products. Data Warehousing for an example, grants me access to a MPP system versus an SMP system to where I can have a columnstore that scales out across dozens of virtual machines versus scaling up on one virtual machine. This is why articles like this are great for people like me. It means we also have the ability to take advantage of Polybase and connect multiple data sources from multiple source systems to a structured data model through external tables.

It's just another tool in the toolbox, not a replacement for an existing tool such as SQL Server 2016.

MarkHinderliter
MarkHinderliter
SSC Rookie
SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)

Group: General Forum Members
Points: 40 Visits: 45
Could use more info around this:

"CREATE EXTERNAL TABLE prices( Id int, product VARCHAR(64), price int) WITH ( LOCATION = '/', DATA_SOURCE = prices, FILE_FORMAT = sqlservercentralformat)"

This, "LOCATION = '/', ", doesn't work. I tried looking it up to see what the location should be and it's pretty cryptic. I suspect this is a path with the container name, blob name, and file name, but when I put in the name of my folder and the file name, LOCATION = '/polybasetutorial/prices.csv', it doesn't work.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum









































































































































































SQLServerCentral


Search