SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Split String in SSIS


Split String in SSIS

Author
Message
pshvets
pshvets
Old Hand
Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)

Group: General Forum Members
Points: 394 Visits: 154
Hello all,
I have a table where one column contains comma delimited string
ID vlaue_string
1 23:45:355:68
1 4545:908:24
2 343:32
3 812:321:23434:34:45:68
There is maximum number of tokens in the string - 6

I need to split each string and put every token in its respective column for further manipulation:
ID vlaue_string Col1 Col2 Col3 Col4 Col5 Col6
1 23:45:35:68 23 45 35 68 null null
1 15:908:24 15 908 24 null null null
2 343:32 343 32 null null null null
3 812:321:434:34:45:68 812 321 434 34 45 68

Could someone please help me to do it?
Is it possible at all to do it in SSIS, or in T-SQL? Or I have to do all processing in CLR?


Thanks a lot in advance!

Pit.
Jack Corbett
  Jack Corbett
SSC-Forever
SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)

Group: General Forum Members
Points: 43519 Visits: 14925
It can be done in SSIS, but you have to use a Script Component to do it. Within the Script Component you would use the Split function which creates an array then you can add the columns to the Script Components output by looping through the array.

You can also do it in T-SQL like this:


DECLARE @table TABLE (ID INT, value VARCHAR(100))

INSERT INTO @table (
ID,
value
)
SELECT
1, '23:45:355:68'
Union All
Select
1, '4545:908:24'
Union All
Select
2, '343:32'
Union All
Select
3, '812:321:23434:34:45:68'

/*
this builds a tally or numbers cte to help do the split
see this article http://www.sqlservercentral.com/articles/TSQL/62867/
for an explanation of the tally table and the split function that follows
*/
;WITH cteTally AS
(
SELECT TOP 100
ROW_NUMBER() OVER(ORDER BY NAME) as n
FROM
sys.all_columns AS AC
),
cteData As
(
SELECT
/*
I need this to uniquely identitfy each row since your
test data includes 2 rows with id 1. If the id column
is unique then you can skip this step and replace value_id
with id in later steps
*/
ROW_NUMBER() OVER (ORDER BY T.id) AS value_id,
T.ID,
T.value
FROM
@table T
),
/*
this splits the values into multiple rows identifying each row
for each value_id for use to PIVOT the data
*/
cteValues AS
(
SELECT
ROW_NUMBER() OVER (PARTITION BY T.value_id ORDER BY T.id) AS row_id,
T.value_id,
T.ID,
SUBSTRING(':'+T.value+':',N+1,CHARINDEX(':',':'+T.value+':',N+1)-N-1) AS split_value,
T.value
FROM
cteData AS T CROSS JOIN
cteTally AS TL
WHERE
N < LEN(':'+T.value+':') AND
SUBSTRING(':'+T.value+':',N,1) = ':'
)
/*
Here is where we return the final data in the format you want,
I used the MIN function but you could use MAX as each row will only
have one value for each column. See this article for details on the
pivoting http://www.sqlservercentral.com/articles/T-SQL/63681/
The first example uses the PIVOT function the second (Commented out) does not.
*/
SELECT
id,
value,
[1] AS Col1,
[2] AS Col2,
[3] AS Col3,
[4] AS Col4,
[5] AS Col5,
[6] AS Col6
FROM
(SELECT value_id, row_id, id, value, split_value FROM cteValues) AS Src
PIVOT (MIN(split_value) FOR row_id IN ([1],[2],[3], [4], [5], [6]))AS pvt
ORDER BY
id,
value

/*
SELECT
id,
value,
Min(CASE
WHEN row_id = 1 THEN split_value
ELSE NULL
END) AS col1,
Min(CASE
WHEN row_id = 2 THEN split_value
ELSE NULL
END) AS col2,
Min(CASE
WHEN row_id = 3 THEN split_value
ELSE NULL
END) AS col3,
Min(CASE
WHEN row_id = 4 THEN split_value
ELSE NULL
END) AS col4,
Min(CASE
WHEN row_id = 5 THEN split_value
ELSE NULL
END) AS col5,
Min(CASE
WHEN row_id = 6 THEN split_value
ELSE NULL
END) AS col6
FROM
cteValues
GROUP BY
value_id,
id,
value
*/



Which you use (SSIS or T-SQL) depends on what you need to do next.



Jack Corbett

Applications Developer

Don't let the good be the enemy of the best. -- Paul Fleming
At best you can say that one job may be more secure than another, but total job security is an illusion. -- Rod at work

Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question
How to Post Performance Problems
Crosstabs and Pivots or How to turn rows into columns Part 1
Crosstabs and Pivots or How to turn rows into columns Part 2
pshvets
pshvets
Old Hand
Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)Old Hand (394 reputation)

Group: General Forum Members
Points: 394 Visits: 154
Thanks a lot, Jack!
It all worked perfect! And performance with Tally table is great!

Thanks a lot for your help!

Pit.
Jack Corbett
  Jack Corbett
SSC-Forever
SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)

Group: General Forum Members
Points: 43519 Visits: 14925
Glad I could help. A tally table has a lot of uses, almost anytime you think you need a loop see if there is a way to use a tally table.



Jack Corbett

Applications Developer

Don't let the good be the enemy of the best. -- Paul Fleming
At best you can say that one job may be more secure than another, but total job security is an illusion. -- Rod at work

Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question
How to Post Performance Problems
Crosstabs and Pivots or How to turn rows into columns Part 1
Crosstabs and Pivots or How to turn rows into columns Part 2
oddiejbp
oddiejbp
SSC Rookie
SSC Rookie (49 reputation)SSC Rookie (49 reputation)SSC Rookie (49 reputation)SSC Rookie (49 reputation)SSC Rookie (49 reputation)SSC Rookie (49 reputation)SSC Rookie (49 reputation)SSC Rookie (49 reputation)

Group: General Forum Members
Points: 49 Visits: 27
Using XML its easier

http://www.sqlservercentral.com/articles/XML/61618/

regards.
CozyRoc
CozyRoc
SSCertifiable
SSCertifiable (6.5K reputation)SSCertifiable (6.5K reputation)SSCertifiable (6.5K reputation)SSCertifiable (6.5K reputation)SSCertifiable (6.5K reputation)SSCertifiable (6.5K reputation)SSCertifiable (6.5K reputation)SSCertifiable (6.5K reputation)

Group: General Forum Members
Points: 6490 Visits: 2235
oddiejbp (10/28/2009)
Using XML its easier

http://www.sqlservercentral.com/articles/XML/61618/

regards.


This approach would worsen the load performance. If you have to load simple CSV file, the much better approach is to use directly the standard bcp facility.

---
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/


Matt Rigbye-353351
Matt Rigbye-353351
Valued Member
Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)

Group: General Forum Members
Points: 73 Visits: 15
Jack Corbett (3/19/2009)
It can be done in SSIS, but you have to use a Script Component to do it. Within the Script Component you would use the Split function which creates an array then you can add the columns to the Script Components output by looping through the array.


Hi Jack - would please be able to provide an example on how to do this using the Scrip Component?
I'm trying to accomplish the same thing, but I so far I have never needed to use that Component before.

Thanks
/Matthew
Jack Corbett
  Jack Corbett
SSC-Forever
SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)

Group: General Forum Members
Points: 43519 Visits: 14925
Matthew,

Here's an example of what you could do. In the example I don't pivot the data as I just want to split the string and have multiple rows. You should be able to figure out how to Pivot it if that is your need:

Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim RowId As String = Row.RowID
Dim DelimitedList As String = Row.DelimitedList
Dim delimiter As String = ","

If Not String.IsNullOrEmpty(RowId.Trim) Then
If Not (String.IsNullOrEmpty(DelimitedList)) Then
Dim DelimitedListArray() As String = DelimitedList.Split(New String() {delimiter}, StringSplitOptions.RemoveEmptyEntries)

For Each item As String In DelimitedListArray
With Output0Buffer
.AddRow()
.RowId = RowId
.Item = item.Replace(ControlChars.Cr, "").Replace(ControlChars.Lf, "").Replace(ControlChars.Tab, "").Trim()
End With
Next
End If
End If
End Sub





Jack Corbett

Applications Developer

Don't let the good be the enemy of the best. -- Paul Fleming
At best you can say that one job may be more secure than another, but total job security is an illusion. -- Rod at work

Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question
How to Post Performance Problems
Crosstabs and Pivots or How to turn rows into columns Part 1
Crosstabs and Pivots or How to turn rows into columns Part 2
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search