SQLServerCentral Article

Replacing BCP with SQLBulkLoad

,

The old good BCP easily goes to the history. XML became worldwide adopted standard for data

interchange. So if you are using BCP for import text data files, and you want to use XML import files instead of text files,

now is good time for change.

At first, we can see BCP utility called from xp_cmdshell procedure partial example:

DECLARE @cmd VARCHAR(256)
SET @cmd='bcp ' + @DbName + '..' + @TableName + ' in ' + @FullImportFileName + ' /S' + @ServerName + ' /f' + @FullFmtFileName + ' -T'
EXEC @Result_cmd = master..xp_cmdshell  @cmd BCP

Advantages: 

  • BCP enable  fast import for big data files
  • Database can import data without developing other application using BCP. 

Although this method for importing data is very fast, it has several limitations when we want to use themin complex data processing systems in integration with other user’s applications:

  • BCP is not appropriate for importing XML data
  • Inattention using master..xp_cmdshell can seriously endanger SQL Server

    security

BCP disable full control over transactions from application to the final insert in database table.

BCP was a very good utility that can help to build application independent database. What does it

mean? I want see clear border between application and database. Databases have to be independent in sense that every action from application against database

can be “Execute Procedure”. I don’t want any SQL statement from application code acting directly in database risking to damage database logic.

In that case applications bugs will have less damage effects in database.

This is especially important if you want clear developers responsibility in your development team.

In case of unexpected crash of application some procedure can be executed by hand through SQL Query Analyzer.

This is reason because the task for importing data in Database I want to grant to the Database itself instead of

some application or user interface.

Then we need appropriate tool for import XML files in SQL Server database, called from Stored Procedure. Although OPENXML statement can be used for direct import in

database, I prefer this option:

Using SQLXMLBulkLoad.SQLXMLBulkLoad.3.0

On your SQL Server you have to install SQLXML3.0 SP1 (http://msdn.microsoft.com),

then create the file ImportData.xml in 'C:\Folder\ImportData.xml' ImportData.xml

using the following data:

<ImportData>
<Row>
<Field1>Row1_ Filed1_Data</Field1>
<Field2>Row1_ Filed2_Data</Field2>
<Field3>Row1_ Filed3_Data</Field3>
</Row>
   <Row>
   <Field1>Row2_ Filed1_Data</Field1>
   <Field2>Row2_ Filed2_Data</Field2>
   <Field3>Row2_ Filed3_Data</Field3>
   </Row>
</ImportData>

You also need to create a file called schema.xml in the same folder, as

follows:

<?xml version="1.0" ?>
<Schema xmlns="urn:schemas-microsoft-com:xml-data" 
        xmlns:dt="urn:schemas-microsoft-com:xml:datatypes"  
        xmlns:sql="urn:schemas-microsoft-com:xml-sql">
   <ElementType name="Field1" dt:type="string"/>
   <ElementType name="Field2" dt:type="string"/>
   <ElementType name="Field3" dt:type="string"/>
   <ElementType name="ElementRow" sql:is-constant="1">
   <element type="Row"/>
   </ElementType>
   <ElementType name="Row"  sql:relation="TableImport">
   <element type="Field1"  sql:field="TabField1"/>
        <element type="Field2"  sql:field="TabField2"/>
        <element type="Field3"  sql:field="TabField3"/>
   </ElementType>
</Schema>

Create TableImport1 in “YourDatabase” executing the following script SQL Query Analyzer:

CREATE TABLE [dbo].[TableImport1] (
[TabField1] [varchar] (40) NULL,
[TabField2] [varchar] (40) NULL ,
[TabField3] [varchar] (40) NULL) ON [PRIMARY]
GO

Then you can create new procedure in you database:

CREATE PROCEDURE BulkLoad AS 
DECLARE@objectINT
DECLARE @hr INT
DECLARE @src VARCHAR(255)
DECLARE @desc VARCHAR (255)
DECLARE @Schema VARCHAR(128)DECLARE @ImportDataVARCHAR(128)
DECLARE @ErrorFile VARCHAR(128)
SET @Schema = 'C:\Folder\schema.xml'SET @ImportData =  'C:\Folder\ImportData.xml'SET @ErrorFile =
'C:\Folder\Error.log'
EXEC @hr = sp_OACreate 'SQLXMLBulkLoad.SQLXMLBulkLoad.3.0', @object OUT
IF @hr <> 0BEGIN
EXEC sp_OAGetErrorInfo @object, @src OUT, @desc OUT
SELECT hr=convert(varbinary(4),@hr), Source=@src, Description=@desc   RETURNEND
ELSE
EXEC @hr = sp_OASetProperty @object, 'ConnectionString', 'provider=SQLOLEDB.1;data source=SERVERNAME; database= YourDatabase;Trusted_Connection=Yes'
IF @hr <> 0
BEGIN
PRINT 'ERROR sp_OAMethod - ConnectionString'                                      EXEC sp_OAGetErrorInfo @object
RETURN
END
EXEC @hr = sp_OASetProperty @object, 'ErrorLogFile', @ErrorFile
IF @hr <> 0
    BEGIN
    PRINT 'ERROR sp_OAMethod - ErrorLogFile'
    EXEC sp_OAGetErrorInfo @object
    RETURN
    END
   
   EXEC @hr = sp_OAMethod @object, 'Execute', NULL, @Schema, @ImportData
IF @hr <> 0
BEGIN
PRINT 'ERROR sp_OAMethod - Execute'
EXEC sp_OAGetErrorInfo @object
RETURN
END
EXEC @hr = sp_OADestroy @object
IF @hr <> 0
BEGIN
PRINT 'ERROR sp_OADestroy '
EXEC sp_OAGetErrorInfo @object
RETURN
END
   SELECT  'OK'
GO

At last you are prepared to execute your procedure to import data into TableImport1

by executing the 'BulkLoad' procedure from SQL Query Analyzer. Don’t forget to change connection string appropriate to your

server (the one shown uses a trusted connection, but you could change to use a

sql login). After executing you can test if it was successful by doing a select

* from TableImport1, which should yield three rows.

Rate

5 (1)

You rated this post out of 5. Change rating

Share

Share

Rate

5 (1)

You rated this post out of 5. Change rating