Stairway to U-SQL Level 6: U-SQL Indexes and Partition Schemes

  • Comments posted to this topic are about the item Stairway to U-SQL Level 6: U-SQL Indexes and Partition Schemes

  • I need to go back and read the prior lessons.

  • We have changed the syntax for Partitioning and Distribution. Can you please update your samples.

    More details here: https://github.com/Azure/AzureDataLake/blob/master/docs/Release_Notes/2016/2016_08_01/USQL_Release_Notes_2016_08_01.md#start-of-deprecation-of-old-partitioned-by-syntax

    Cheers

    Michael

  • Hi Mike

    Thanks very much for your updates. I'll take a look and will update this article in the next week or so.

    Regards,

    Mike.

  • For 070 script, it says file not found: /ssc_uk_postcodes/postcodes/postcodes.csv

    I am following the series from start and have the project built on level 5. Thanks.

  • nzhang6666 (10/14/2016)


    For 070 script, it says file not found: /ssc_uk_postcodes/postcodes/postcodes.csv

    I am following the series from start and have the project built on level 5. Thanks.

    It works now, but I change the path to where the postcode file saved locally, something like this:

    DECLARE @DataFilePath string = "C:/Users/n.z/Downloads/usql_files/U-SQL_Files/Postcodes/Postcodes.csv";

    also the output file path, something like this: TO "C:/Users/n.z/Downloads/usql_files/U-SQL_Files/Postcodes/knowsleyPostcodes.tsv"

    Also found another way is to create folders under project, e.g. for the original script to work, need to create folders ssc_uk_postcodes/postcodes under project, and add the postcode csv in it. The output location will be (automatically) C:\Users.z\AppData\Local\USQLDataRoot\outputs

  • Hi nzhang6666

    Good to hear you are following along, and sorry to hear you had some trouble with the files. I've just had a check and I've noticed that I don't specify how to add the files to the Visual Studio project in Level 5. I'll update the article to reflect this.

    To follow the series exactly, you need to create a ssc_uk_postcodes folder within your Visual Studio solution, with appropriate sub-folders (counties, postcodes, districts, postcodeestimates). Then put each of the appropriate files in there. Which is what you mention in your second point.

    Your change (where you've specified a local file path) will work locally, but it won't work if you try to run the code in Azure. Adding the above folders locally and in Azure will guarantee consistency in the scripts, as you're using relative instead of absolute paths.

    I'll post a message here once the article has been updated. Thanks for your interest.

    Best regards,

    Mike.

  • I just started on U-SQL and new to Microsoft Azure. Quick basic Question - If I need to insert any Data into U-SQL Database, first I need to copy over the contents of the Table from OnPremise Server to Azure Data Lake Store and then use U-SQL Script to insert the Data into a Indexed, Partitioned Table in U-SQL Database. I think I am having multiple Files for the same Data set when it comes to Azure Storage. Can we insert Data into U-SQL Tables (U-SQL Database) directly from onPremise Server ???

  • Hi Guna

    Sorry for my delayed reply. Unfortunately you must copy your data up to the Data Lake first before you can insert data.

    The only alternative is to run your database locally via Visual Studio and run queries from there. Of course, you're missing out on the benefits of using Azure doing that.

    Regards,
    Mike.

Viewing 9 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic. Login to reply