Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Bulk Inserting Comma separated Quotation Delimited text file Expand / Collapse
Author
Message
Posted Sunday, January 13, 2013 4:48 PM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Wednesday, May 15, 2013 1:00 PM
Points: 62, Visits: 109
I am receiving a text file with comma separated data with quotation marks around certain fields that I need to bulk insert into SQL Server 2008 R2. An example of the data is:

"BUNGALOW 101,RIDGEWAY DR","POINT FORTIN",,,"TT","868-389-9769",,"ZPXB303191",1

As you can see there is a comma in the first field.

Using this Bulk Insert command causes that first field to be split into two columns. I know BOL says this is what is supposed to happen and to choose the separators carefully, but I have no control over the text file which is created by a 20 year old system which cannot be changed.

Any ideas how I can import this?
Post #1406531
Posted Sunday, January 13, 2013 6:44 PM


SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Yesterday @ 7:24 PM
Points: 2,346, Visits: 3,192
I'm not sure if this will help you, but the following code will replace the comma embedded between quotation marks (as long as it doesn't appear in the first position) with a semicolon.

;WITH MyData (ID, strcol) AS (
SELECT 1, '"BUNGALOW 101,RIDGEWAY DR","POINT FORTIN",,,"TT","868-389-9769",,"ZPXB303191",1'),
NewData (ID, ItemNumber, strcol) AS (
SELECT ID, ItemNumber
,strcol=CASE WHEN CHARINDEX(',', Item) > 1 THEN REPLACE(Item, ',', ';') ELSE Item END
FROM MyData
CROSS APPLY PatternSplitCM(strcol, '["]'))
SELECT ID, (
SELECT strcol + ''
FROM NewData b
WHERE a.ID = b.ID
ORDER BY ItemNumber
FOR XML PATH(''))
FROM NewData a
GROUP BY ID


The PatternSplitCM FUNCTION can be found in the 4th article linked into my signature line.

You would need to decide what character was appropriate to use in place of semicolon, then REPLACE it back to comma after BULK INSERT completes its task (if that is possible).



No loops! No CURSORs! No RBAR! Hoo-uh!

INDEXing a poor-performing query is like putting sugar on cat food. Yeah, it probably tastes better but are you sure you want to eat it?

Need to UNPIVOT? Why not CROSS APPLY VALUES instead?
Since random numbers are too important to be left to chance, let's generate some!
Are you too recursively challenged?
Splitting strings based on patterns can be fast!
Post #1406538
Posted Sunday, January 13, 2013 7:23 PM


Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Yesterday @ 11:18 AM
Points: 364, Visits: 580
SSIS would handle this very nicely and be easily repeatable.
Post #1406541
Posted Sunday, January 13, 2013 8:39 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: General Forum Members
Last Login: Yesterday @ 4:51 PM
Points: 32,923, Visits: 26,811
jerry-621596 (1/13/2013)
SSIS would handle this very nicely and be easily repeatable.


Agreed. ACE drivers with OPENROWSET would also work well.


--Jeff Moden
"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

For better, quicker answers on T-SQL questions, click on the following...
http://www.sqlservercentral.com/articles/Best+Practices/61537/

For better answers on performance questions, click on the following...
http://www.sqlservercentral.com/articles/SQLServerCentral/66909/
Post #1406552
Posted Monday, January 14, 2013 4:47 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Wednesday, May 15, 2013 1:00 PM
Points: 62, Visits: 109
Thanks Dwain,

I'll try that out
Post #1406654
Posted Monday, January 14, 2013 4:48 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Wednesday, May 15, 2013 1:00 PM
Points: 62, Visits: 109
Sorry, I should have said this was SQL Server 2008 R2 Express. I don't think SSIS is available.
Post #1406655
Posted Monday, January 14, 2013 4:49 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Wednesday, May 15, 2013 1:00 PM
Points: 62, Visits: 109
Jeff Moden (1/13/2013)
jerry-621596 (1/13/2013)
SSIS would handle this very nicely and be easily repeatable.


Agreed. ACE drivers with OPENROWSET would also work well.


Thanks, I'll research that.
Post #1406656
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse