Post reply

Need Help: Polybase external table on Parquet file stored in Azure StorageV2

  • April 2, 2024 at 8:23 pm

    #4406618

    My goal is to create a Polybase external table from a Parquet file located in an Azure storage container.

    • SQL Server 2019 (RTM-CU22-GDR) (KB5029378) - 15.0.4326.1 (X64)
    • StorageV2 (general purpose v2) (LRS)
    • File was exported from SQL Server using: parquet-cpp-arrow version 4.0.0

    I know that the SAS token I'm using is probably good in terms of start/end date, allowed permissions/services/resource types,  IP firewall, etc. because I've used the same token to unit test uploads and downloads of the file programmatically in PowerShell.

    Here is the code I'm using so far in an attempt to create the external table:

    CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<MyMasterKey>';
    GO

    CREATE DATABASE SCOPED CREDENTIAL ArchiveCredential
    WITH IDENTITY = '<MyStorageAccount>',
    SECRET = '<MyAccountKey>';
    GO

    CREATE EXTERNAL DATA SOURCE ArchiveDataSource
    WITH (
        LOCATION = 'wasbs://<MyContainer>@<MyStorageAccount>.blob.core.windows.net/',
        CREDENTIAL = ArchiveCredential,
    TYPE = HADOOP
    );
    GO

    CREATE EXTERNAL FILE FORMAT ArchiveFormat
    WITH ( FORMAT_TYPE = PARQUET );
    GO

    CREATE EXTERNAL TABLE CitiesArchive
    (
    CityID INT NOT NULL,
    CityName VARCHAR(50) NOT NULL
    )
    WITH (
        LOCATION = '/parquet/CitiesArchive.parquet',
        DATA_SOURCE = ArchiveDataSource,
        FILE_FORMAT = ArchiveFormat
    );
    GO

    SELECT CityID,
    CityName
    FROM CitiesArchive;
    GO

    When I attempt to SELECT from CitiesArchive, I get the following error message:

    Msg 596, Level 21, State 1, Line 40
    Cannot continue the execution because the session is in the kill state.
    Msg 0, Level 20, State 0, Line 40
    A severe error occurred on the current command.  The results, if any, should be discarded.

    Here is what I'm using as a guide:

    https://learn.microsoft.com/en-us/sql/t-sql/statements/create-external-data-source-transact-sql?view=sql-server-ver15&tabs=dedicated

    Any ideas?

     

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

  • April 3, 2024 at 7:34 pm

    #4407109

    - UPDATE -

    After examining SQL Server error log, I saw the following message:

    A user request from the session with SPID 180 generated a fatal exception. SQL Server is terminating this session. Contact Product Support Services with the dump produced in the log directory.

    I decided the reboot the server, and then attempted again to select from the external table.

    The issue is now resolved!

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

Viewing 2 posts - 1 through 1 (of 1 total)

You must be logged in to reply to this topic. Login to reply