Home Forums Programming XML xml openrowset performance RE: xml openrowset performance

  • First suggestion is to add one step where the files are directly loaded into a staging table as an XML blob, without any shredding or processing. This will give you the actual performance of the file share minus the write performance of the SQL Server. In my experience, 5-10Mb/sec at least. Using the BulkColumn directly has very adverse affects on the performance so lets try to avoid that one.

    From that initial staging table, do the shredding, if you share some more details here we can help you optimize that step.

    On the performance side of things, what you can expect is an overhead of around 50ms for initialization in addition to 5-10Mb/sec transfer rate for each file, somehow there seams to be a barrier there although using SSD type subsystems I've managed to get this much much higher. The shredding part normally is where the low hanging fruits resides so I'm looking forward to analyse your approach and hopefully improve it.

    To sum up, can you post some XML examples and the code you are using to shred those?

    😎