SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Writing variable length records to a flat file destination in SSIS


Writing variable length records to a flat file destination in SSIS

Author
Message
Robert Frasca
Robert Frasca
SSCertifiable
SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)

Group: General Forum Members
Points: 5351 Visits: 832
I have a 132, 180, 18, and 34 column records and I need to write them to a flat file destination. Each record consists of one character string of those lengths. This is a header, detail, special, and trailer record scenario. In the Advance tab of the Flat File Connection Manager it wants an OutputColumnWidth. I have set it to 180 because that's the longest record but my client is rejecting the file because the header record is too long. I have searched high and low and I can't find anything that will tell me how to write variable length records to a file unless the file is delimited. I'm not really that C# savvy except for the relatively simple scripting I do in SSIS. Could I do it with C# or some other output mechanism?

"Beliefs" get in the way of learning.
drew.allen
drew.allen
SSC Guru
SSC Guru (67K reputation)SSC Guru (67K reputation)SSC Guru (67K reputation)SSC Guru (67K reputation)SSC Guru (67K reputation)SSC Guru (67K reputation)SSC Guru (67K reputation)SSC Guru (67K reputation)

Group: General Forum Members
Points: 67373 Visits: 17799
You treat your output as a single field and create a delimited file. The delimiter separates fields and there are always one fewer delimiters than fields, so if you are only outputting one field, you have one fewer delimiters or zero delimiters.

Drew

J. Drew Allen
Business Intelligence Analyst
Philadelphia, PA
How to post data/code on a forum to get the best help.
How to Post Performance Problems
Make sure that you include code in the appropriate IFCode tags, e.g. [code=sql]<your code here>[/code]. You can find the IFCode tags under the INSERT options when you are writing a post.
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)

Group: General Forum Members
Points: 221390 Visits: 25150
Robert Frasca - Wednesday, February 28, 2018 7:15 AM
I have a 132, 180, 18, and 34 column records and I need to write them to a flat file destination. Each record consists of one character string of those lengths. This is a header, detail, special, and trailer record scenario. In the Advance tab of the Flat File Connection Manager it wants an OutputColumnWidth. I have set it to 180 because that's the longest record but my client is rejecting the file because the header record is too long. I have searched high and low and I can't find anything that will tell me how to write variable length records to a file unless the file is delimited. I'm not really that C# savvy except for the relatively simple scripting I do in SSIS. Could I do it with C# or some other output mechanism?

One option would be to create four separate files and then put them all together at the end (a simple DOS COPY command can do this).



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Robert Frasca
Robert Frasca
SSCertifiable
SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)

Group: General Forum Members
Points: 5351 Visits: 832
Okay, I resolved this in such a simple fashion that I'm embarrassed that I didn't think of it sooner. Each record has a type number, i.e. 1 for header, 2 for detail etc.

SELECT RRE,
ClaimantID,
RowType,
TextData
FROM ( SELECT
RRE,
ClaimantID,
RowType,
CASE (RowType)
WHEN 1 THEN SUBSTRING(TextData,1,132)
WHEN 2 THEN SUBSTRING(TextData,1,180)
WHEN 3 THEN SUBSTRING(TextData,1,18)
WHEN 4 THEN SUBSTRING(TextData,1,32)
END AS TextData
FROM dbo.TMP_ClaimValidationFinalTextData
) AS Y
ORDER BY ClaimantID, RowType
;


"Beliefs" get in the way of learning.
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)

Group: General Forum Members
Points: 221390 Visits: 25150
Robert Frasca - Wednesday, February 28, 2018 9:16 AM
Okay, I resolved this in such a simple fashion that I'm embarrassed that I didn't think of it sooner. Each record has a type number, i.e. 1 for header, 2 for detail etc.

SELECT RRE,
ClaimantID,
RowType,
TextData
FROM ( SELECT
RRE,
ClaimantID,
RowType,
CASE (RowType)
WHEN 1 THEN SUBSTRING(TextData,1,132)
WHEN 2 THEN SUBSTRING(TextData,1,180)
WHEN 3 THEN SUBSTRING(TextData,1,18)
WHEN 4 THEN SUBSTRING(TextData,1,32)
END AS TextData
FROM dbo.TMP_ClaimValidationFinalTextData
) AS Y
ORDER BY ClaimantID, RowType
;

Nice and simple.
Just wondering ... did you choose SUBSTRING() rather than LEFT() for any particular reason?



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Robert Frasca
Robert Frasca
SSCertifiable
SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)SSCertifiable (5.4K reputation)

Group: General Forum Members
Points: 5351 Visits: 832
Phil Parkin - Wednesday, February 28, 2018 9:22 AM
Robert Frasca - Wednesday, February 28, 2018 9:16 AM
Okay, I resolved this in such a simple fashion that I'm embarrassed that I didn't think of it sooner. Each record has a type number, i.e. 1 for header, 2 for detail etc.

SELECT RRE,
ClaimantID,
RowType,
TextData
FROM ( SELECT
RRE,
ClaimantID,
RowType,
CASE (RowType)
WHEN 1 THEN SUBSTRING(TextData,1,132)
WHEN 2 THEN SUBSTRING(TextData,1,180)
WHEN 3 THEN SUBSTRING(TextData,1,18)
WHEN 4 THEN SUBSTRING(TextData,1,32)
END AS TextData
FROM dbo.TMP_ClaimValidationFinalTextData
) AS Y
ORDER BY ClaimantID, RowType
;

Nice and simple.
Just wondering ... did you choose SUBSTRING() rather than LEFT() for any particular reason?

I was thinking about strings and SUBSTRING was the first to come to mind. Is LEFT more efficient?


"Beliefs" get in the way of learning.
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)SSC Guru (221K reputation)

Group: General Forum Members
Points: 221390 Visits: 25150
Robert Frasca - Wednesday, February 28, 2018 10:05 AM
Phil Parkin - Wednesday, February 28, 2018 9:22 AM
Nice and simple.
Just wondering ... did you choose SUBSTRING() rather than LEFT() for any particular reason?

I was thinking about strings and SUBSTRING was the first to come to mind. Is LEFT more efficient?

I doubt that there is any difference in terms of performance (though I have not checked that).
I mentioned it because
LEFT(TextData,132)
is equivalent to
SUBSTRING(TextData,1,132)
and requires seven fewer characters and one fewer function argument, making it somewhat easier on the eye (in my opinion).



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum









































































































































































SQLServerCentral


Search