Writing to a File Using the sp_OACreate Stored Procedure and OSQL: The
-
Comments posted to this topic are about the content posted at http://www.sqlservercentral.com/columnists/zNichter/writingtoafileusingthesp_oacreatestoredprocedurean.asp
--------------------------
Zach -
Excellent article.
It's always useful to know just how the server performs under different situations.
I tend to use Perl with a pipe from OSQL to access data exported from the SQL server (which allows filtering/formatting on the fly if necessary) as a default method of handling data output to a file.
There have been times when I've been tempted to keep the file writing on the server side, to aid reliability of write for certain actions, but this article will definitely make me think twice before using the SP methods for file creation.
Thanks for a good read!
-
I would generally pump the performance data to a table. This allows a lot more flexibility in the reporting. If you need some kind of report you can use a reporting package or even just DTS it out to a text file.
Your article had an excellent conclusion. Based on it's title, I really expected a pro-sp_OAx article but was very pleasantly suprised. It's great to see someone taking time out to make decisions based on solid data and I think this sets an excellent example for the novices that come here to learn.
Kudos!
[font="Tahoma"]Bryant E. Byrd, BSSE MCDBA MCAD[/font]
Business Intelligence Administrator
MSBI Administration Blog -
Nice article, thanks for sharing your research!
-
Very Nice article. Covered a lot of ground and it's a pleasure to actually see commented formatted code. I've seen a method to write to text files using linked servers through the use of a file called Schema.ini (I think that's what it's called). Sure would be nice to see a follow-up article on that.
Like someone said before me, "Thanks for sharing your research!"
--Jeff Moden
RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.Change is inevitable... Change for the better is not.
Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally) -
Thanks for all of the responces on the article. I'll have to look into the schema.ini option I'll let you know if I find anything out on it.
I wish I knew Perl, I've been wanting to pick up book on it for some time but have never gotten around to it.
Thanks Again,
Zach
--------------------------
Zach -
Where was the example using bcp or DTS?
If you're going to use a hammer to drive a screw, be thorough enough to examine the rest of the non-screwdriver solutions.
-
I agree that it is too bad that you did not include bcp and DTS in your analysis. For a simple write to file, though, DTS seems hardly appropriate. And when you're trying avoid the dreaded xp_cmdshell in a user application, then the cost xp_OACreate is much more tolerable. However, I never realized how big of a cost xp_OACreate was. Thanks for the article!
-
Mike, thanks for the responce. The goal of the article was more to identify the performance factor involved in the use of the sp_oa extended procs and using it and other meathods through Query analyzer to identify the overhead associated to them. But you are right though, bcp is a much better option, and as for DTS, it really didn't fit the need of the situation that I was in. Maybe you should consider contributing to the community yourself sometime, it might make you a little less grumpy...
Oh and thanks for the comments Charles.
--------------------------
Zach -
I believe this was an unfair comparison against sp_OACreate.
Test2, Test3, and Test4 all open the file first, then write the 100,000 lines, then close the file.
Test5 and Test6 100,000 times does this: creates the object, opens the file, writes a line, and then destroys the object (which I am assuming close the file handle?).
If the same flow (open-write-close) was done on the other files 100k times, I am sure they would seem slower also. A better comparison of sp_OACreate would be to create, open, write 100k, close, then destroy.
But then again it feels like Monday morning... Am I missing something about these tests?
-
Rick you are right about Test5 (see below for Test6 comments) the (open-write-close) does contribute to all of the additional overhead there, and it really isn't a valid comparison to all of the other tests. It was added because it was part of the learning process with writing to multiple files for multiple collects and then trying to wrap that in an over all waitfor.
Test6 though does open once, write 100,000 times then close and still the duration was 67 sec, so I think this is a valid comparison.
Let me know if you disagree, I'm always up for learning more.
Thanks for the comment.
--------------------------
Zach -
Yet another approach is to write to a table, then use sp_OA... stored procedure to create a BulkCopy object and "export" the table to a file. I got this method from K Henderson's Guru's Guide to SQL Server Stored Procedures... He includes a procedure, sp_ExportTable that performs this operation very quickly (using speed of BulkCopy) and avoids xp_cmdshell altogether. I am curious how its speed compares--might have to test it.
-
Great! Thanks Charles I'll take a look.
--------------------------
Zach -
OK sorry, I was not intending to be truculent.
It seemed to me that you were writing as an authority, who would have had an editor review your submission for completeness. Had I understood that you were sharing the process of discovering a significant performance difference between two methods (regardless of other options) - I would have taken a much different tone in my comment.
Thanks for pointing out that sqlservercentral.com articles are community-driven. What I have read to date made me believe they're authored by paid writers - so in that vein, keep up the good work.
-
From experience: The print command from T-sql has degrading performance as the file grows.
Optimal solution may be dependant upon filesize of written file.
Viewing 15 posts - 1 through 15 (of 32 total)
You must be logged in to reply to this topic. Login to reply