Loading XML Data into SQL Server (SQL Spackle)

  • WayneS

    SSC Guru

    Points: 95341

    Comments posted to this topic are about the item Loading XML Data into SQL Server (SQL Spackle)

    Wayne
    Microsoft Certified Master: SQL Server 2008
    Author - SQL Server T-SQL Recipes


    If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!
    Links:
    For better assistance in answering your questions
    Performance Problems
    Common date/time routines
    Understanding and Using APPLY Part 1 & Part 2

  • Jeff Moden

    SSC Guru

    Points: 994677

    I have to admit, I've not yet had to enter the realm of loading XML files. But now I know where to go for a quick reference. Thanks for filling in the cracks on this one, Wayne!

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
    "If you think its expensive to hire a professional to do the job, wait until you hire an amateur."--Red Adair
    "Change is inevitable... change for the better is not."
    When you put the right degree of spin on it, the number 3|8 is also a glyph that describes the nature of a DBAs job. 😉

    Helpful Links:
    How to post code problems
    Create a Tally Function (fnTally)

  • Kelsey Thornton

    SSCrazy

    Points: 2157

    WayneS (1/1/2011)


    If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!


    It's a nice article. Can you explain why do you use the OpenRowset function to load the data into a staging table?

    Kelsey Thornton
    MBCS CITP

  • SQL-DBA

    Hall of Fame

    Points: 3004

    I wonder what others are doing about doing an example like this when the XML is from a web service. I run into that more often then XML files. I tend to write C# as a separate application, but are db pros accessing plain old XML web services in the CLR of SQL Server. I would like to see an example like that.

  • WayneS

    SSC Guru

    Points: 95341

    Kelsey Thornton (1/3/2011)


    WayneS (1/1/2011)


    If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!


    It's a nice article. Can you explain why do you use the OpenRowset function to load the data into a staging table?

    Hi Kelsey,

    Very good question. (I especially like the quote from my signature... :-)) In retrospect, I think I should have included this in the article. Thanks for bringing it up!

    To answer your question - it's to load the entire XML file into one record into a table, where we can then work with it. The code in reference is:

    OPENROWSET(BULK N'C:\SQL\cd_catalog.xml', SINGLE_BLOB)

    Notice that we're using the "BULK" and "SINGLE_BLOB" options of OPENROWSET. From the OpenRowset BOL reference in the article, here is what these two options do:

    BULK

    Uses the BULK rowset provider for OPENROWSET to read data from a file.

    SINGLE_BLOB

    Returns the contents of data_file as a single-row, single-column rowset of type varbinary(max).

    I'm sure that there are other ways to accomplish this (for instance, utilize the sp_OA procedures to start up Scripting.FileSystemObject, open the file, read it line by line, append what you just read into a variable, and when finished reading the file, close it and write the results to a table - and clean up all the objects that you had to create along the way). Currently, I believe that using OpenRowset with the BULK and SINGLE_BLOB options is going to be the easiest, and probably the fastest way.

    A discussion of all the various ways to load an external file into a table would lead to an article of it's own - beyond the scope of this "SQL Spackle"! Therefore, I just used the quick/easy way to get the job done.

    Wayne
    Microsoft Certified Master: SQL Server 2008
    Author - SQL Server T-SQL Recipes


    If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!
    Links:
    For better assistance in answering your questions
    Performance Problems
    Common date/time routines
    Understanding and Using APPLY Part 1 & Part 2

  • WayneS

    SSC Guru

    Points: 95341

    SQL-DBA (1/3/2011)


    I wonder what others are doing about doing an example like this when the XML is from a web service. I run into that more often then XML files. I tend to write C# as a separate application, but are db pros accessing plain old XML web services in the CLR of SQL Server. I would like to see an example like that.

    I think that this link will give you an example.

    Wayne
    Microsoft Certified Master: SQL Server 2008
    Author - SQL Server T-SQL Recipes


    If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!
    Links:
    For better assistance in answering your questions
    Performance Problems
    Common date/time routines
    Understanding and Using APPLY Part 1 & Part 2

  • Kelsey Thornton

    SSCrazy

    Points: 2157

    Thanks for this.

    It's now clearer.

    Kelsey Thornton
    MBCS CITP

  • paul.marsh

    SSC Rookie

    Points: 41

    decent article - good reference. but I feel a comment about the bcp utility should be added for those situations that warrant a different approach.

  • Steve Jones - SSC Editor

    SSC Guru

    Points: 715889

    Excellent discussion, Wayne. This is one that I will keep bookmarked.

  • gregg_dn

    Ten Centuries

    Points: 1193

    I'm getting an the following error when I try to run the OPENROWSET function:

    DECLARE @CD TABLE (XMLData XML);

    INSERT INTO @CD

    SELECT *

    FROM OPENROWSET(BULK N'C:\SQL\cd_catalog.xml', SINGLE_BlOB)

    Msg 491, Level 16, State 1, Line 5

    A correlation name must be specified for the bulk rowset in the from clause.

    Is there something in the code beyond SINGLE_BLOB) ?

    Thanks

    G. Daniels

  • shad-873858

    SSC Rookie

    Points: 30

    gregg_dn (1/3/2011)


    I'm getting an the following error when I try to run the OPENROWSET function:

    DECLARE @CD TABLE (XMLData XML);

    INSERT INTO @CD

    SELECT *

    FROM OPENROWSET(BULK N'C:\SQL\cd_catalog.xml', SINGLE_BlOB)

    Msg 491, Level 16, State 1, Line 5

    A correlation name must be specified for the bulk rowset in the from clause.

    Is there something in the code beyond SINGLE_BLOB) ?

    Thanks

    G. Daniels

    Yes, as the author noted, you have to provide an alias to the OpenRowset. In his example the author used rs. This will get you past that particular error. Good luck.

  • mtillman-921105

    SSCertifiable

    Points: 7049

    Excellent Wayne, thank you sir!

    :smooooth:

    ______________________________________________________________________The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge. - Stephen Hawking

  • gregg_dn

    Ten Centuries

    Points: 1193

    Thanks!

  • WayneS

    SSC Guru

    Points: 95341

    Kelsey Thornton (1/3/2011)


    Thanks for this.

    It's now clearer.

    Kelsey,

    Good, I'm glad it all makes sense. If it hadn't, I would have failed.

    Wayne
    Microsoft Certified Master: SQL Server 2008
    Author - SQL Server T-SQL Recipes


    If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!
    Links:
    For better assistance in answering your questions
    Performance Problems
    Common date/time routines
    Understanding and Using APPLY Part 1 & Part 2

  • WayneS

    SSC Guru

    Points: 95341

    paul.marsh (1/3/2011)


    decent article - good reference. but I feel a comment about the bcp utility should be added for those situations that warrant a different approach.

    Paul - thanks. But how would you use BCP to load an XML file in? (This is what this article is about...)

    Wayne
    Microsoft Certified Master: SQL Server 2008
    Author - SQL Server T-SQL Recipes


    If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!
    Links:
    For better assistance in answering your questions
    Performance Problems
    Common date/time routines
    Understanding and Using APPLY Part 1 & Part 2

Viewing 15 posts - 1 through 15 (of 61 total)

You must be logged in to reply to this topic. Login to reply