Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Loading XML Data into SQL Server (SQL Spackle)


Loading XML Data into SQL Server (SQL Spackle)

Author
Message
dg81328
dg81328
Grasshopper
Grasshopper (17 reputation)Grasshopper (17 reputation)Grasshopper (17 reputation)Grasshopper (17 reputation)Grasshopper (17 reputation)Grasshopper (17 reputation)Grasshopper (17 reputation)Grasshopper (17 reputation)

Group: General Forum Members
Points: 17 Visits: 160
even though u successfully loaded the raw xml into a table you still processed the data one row at a time. It is much better to use OPENXML and pivot the data.

example:
--TABLE TO HOLD THE RAW XML
DECLARE @tbl TABLE
(
ID INT, ParentID INT, NodeType INT, LocalName NVARCHAR(100), Prefix VARCHAR(50),
NameSpaceURI VARCHAR(50), DataType VARCHAR(50), Prev VARCHAR(50), [TEXT] NVARCHAR(200)
)
--xml in a variable of XML type that might be used in a proc or fnc
EXEC sp_xml_preparedocument @docHandle OUTPUT,@ItemsXML;
INSERT INTO @tbl(
ID ,ParentID ,NodeType ,LocalName,Prefix ,NameSpaceURI ,DataType ,Prev ,[TEXT]
)
SELECT * FROM OPENXML(@docHandle, '/PackageRequests/ItemRequest',1)
EXEC sp_xml_removedocument @docHandle
;



WITH CTE_Ranked AS (
SELECT
T1.LocalName
,T2.Text
,NTILE(12) OVER (PARTITION BY T1.LocalName ORDER BY T1.ID) AS RNKING
--NTILE Count defined by counting the largest number of possible records for any one column
--this could easily be a variable
FROM @tbl T1
INNER JOIN @Tbl T2
ON T1.ID = T2.ParentID
WHERE T1.LocalName IN ('VndID','LocID')
),
CTE_PIVOT_SOURCE AS (
SELECT
CTE_Ranked0.LocalName AS RLN
,CTE_Ranked0.text AS RIV
,CTE_Ranked01.RNKING
FROM CTE_Ranked CTE_Ranked0
INNER JOIN CTE_Ranked CTE_Ranked01 ON CTE_Ranked01.RNKING = CTE_Ranked0.RNKING
)
SELECT VendorID,LocationID
FROM CTE_PIVOT_SOURCE
PIVOT (
MIN(RIV)
FOR RLN IN (VndID,LocID)
) AS PivotTable
jmpeters
jmpeters
SSC Rookie
SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)

Group: General Forum Members
Points: 27 Visits: 34
Responded to wrong post...
jmpeters
jmpeters
SSC Rookie
SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)

Group: General Forum Members
Points: 27 Visits: 34
This took all of five minutes to copy the code, test it out, and save it for future use.

The article was clear, simple, and short. The need to parse XML data is not uncommon for me, but it is infrequent. So instead of reinventing a solution, I may be able to save a few hours here and there by starting with your solution.

Thanks!
Jim Peters
boumerlin
boumerlin
SSC Journeyman
SSC Journeyman (97 reputation)SSC Journeyman (97 reputation)SSC Journeyman (97 reputation)SSC Journeyman (97 reputation)SSC Journeyman (97 reputation)SSC Journeyman (97 reputation)SSC Journeyman (97 reputation)SSC Journeyman (97 reputation)

Group: General Forum Members
Points: 97 Visits: 680
Nice Article! I first started working with XML and .Net web services about 10 years ago as a developer.

These days I wear a DBA hat and just had to write a load process for several XML interfaces. Wish I had seen this first! I found SSIS to be fairly limited and clunky compared to my earlier .Net experiences.

I settled on using the XML Source. I ended up writing a schema validation and XSLT piece to make the XML palatable for the XML Source object. XML Source wasn't very flexible, and certainly wasn't as fast as a bulk load.



Jeff Moden
Jeff Moden
SSC-Forever
SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)

Group: General Forum Members
Points: 45147 Visits: 39923
dg81328 (5/31/2013)
even though u successfully loaded the raw xml into a table you still processed the data one row at a time. It is much better to use OPENXML and pivot the data.


I'm not sure how you can say that so please explain. From what I can see, the methods in the article will allow you to handle multiple XML documents that are loaded into a table (@CD in the article is a table variable) without RBAR whereas the OPENXML document that you suggest can only handle one XML document at a time.

Further, and I admit that I haven't tested it, I suspect that code you posted will be slower because of the PIVOT.

Like I said, please explain your claims about why you think the use of OPENXML and a pivot provides any advantage of the method in the article because I'm just not seeing it and am always interested in better ways.

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
Although they tell us that they want it real bad, our primary goal is to ensure that we dont actually give it to them that way.
Although change is inevitable, change for the better is not.
Just because you can do something in PowerShell, doesnt mean you should. Wink

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
GPO
GPO
Ten Centuries
Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)

Group: General Forum Members
Points: 1054 Visits: 1908
Great article. For those too young to know, Empire Burlesque was a towering collaboration between Mark Knopfler and Bob Dylan that never really got the credit it deserved. That it was wrested back from obscurity into an odd new kind of limelight made the article all the more enjoyable!

:-)

One of the symptoms of an approaching nervous breakdown is the belief that one's work is terribly important.
Bertrand Russell

GPO
GPO
Ten Centuries
Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)Ten Centuries (1.1K reputation)

Group: General Forum Members
Points: 1054 Visits: 1908
Spam reported.

:-)

One of the symptoms of an approaching nervous breakdown is the belief that one's work is terribly important.
Bertrand Russell

WayneS
WayneS
SSCertifiable
SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)

Group: General Forum Members
Points: 6235 Visits: 10403
Jim, boumerlin - Thanks! I'm glad that you were able to add this solution to your toolbox.

Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes
If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!
Links: For better assistance in answering your questions, How to ask a question, Performance Problems, Common date/time routines,
CROSS-TABS and PIVOT tables Part 1 & Part 2, Using APPLY Part 1 & Part 2, Splitting Delimited Strings

Jeff Moden
Jeff Moden
SSC-Forever
SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)SSC-Forever (45K reputation)

Group: General Forum Members
Points: 45147 Visits: 39923
jcasement (5/31/2013)
While it offers a very brief sample to get going fast it should at least list other alternatives since some of them are quite nice. Readers need to be aware of as many alternatives as possible since some technologies have size and performance limitations.

I load massive XML files with SQLXMLBulkload:

Just supply a connection string, a XSD and the XML file and you can load directly into any table.

and it even supports an error log should anything go wrong and you can control it from a client whether it be C# or just a simple VBS like below:

Dim objBL
set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkload.4.0")
objBL.ConnectionString = "Provider=SQLOLEDB.1;Persist Security Info=False;User ID=dbo_Dfrndis;Password=abc1234;Initial Catalog=frndis;Data Source=sqldev"
objBL.ErrorLogFile = "C:\MSSQL\clients\INTLdist\CRD\XLT\error.xml"
objBL.Execute "CRD.xsd", "CRD20130429122127010074ToCRD-2013-05-01_02-59-43-PM.xml"
set objBL=Nothing

Sid


What's the largest file you've loaded with it? And I hope the connection string you just posted doesn't actually contain valid login info.

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
Although they tell us that they want it real bad, our primary goal is to ensure that we dont actually give it to them that way.
Although change is inevitable, change for the better is not.
Just because you can do something in PowerShell, doesnt mean you should. Wink

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
kevinhip
kevinhip
Forum Newbie
Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)

Group: General Forum Members
Points: 3 Visits: 2
Thank you for sharing this wonderful post.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search