SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


load multiple xml file into a single flat file destination


load multiple xml file into a single flat file destination

Author
Message
Nomvula
Nomvula
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1212 Visits: 714
hi guys
i need help, i've created a ForeachLoop Container which loop through all the files .xml in my source directory then created XML Source and flat file Destination, my package execute successfully but create multiple output .txt file instead i one to output in one single .txt file

thanks
Andrew Watson-478275
Andrew Watson-478275
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2546 Visits: 2798
What is the connectionstring set to in the connection manager for the output file? I can't see how you can get multiple files unless you're renaming dynamically during the loop.
Nomvula
Nomvula
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1212 Visits: 714
thank you for the response
here's my output path C:\OnKeyImports\WorkOrderCosting\Source\WorkOrderCosting_20130419_100035.txt

the expression on the connection string is @[User::ImportSourcePath] + @[User::FilePrefix] + "_" + (DT_WSTR,4)DatePart("yyyy", GetDate()) + RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) + "_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]
Andrew Watson-478275
Andrew Watson-478275
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2546 Visits: 2798
Are you getting multiple files with different times in the name, or are you getting a single file which is being overwritten each time round the loop?
Nomvula
Nomvula
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1212 Visits: 714
i'm getting multiple files outputs whereas i want to combine all my input to a single file.

here's what i'm doing. i have multiple files e.g Item1.xml, Item2.xml, Item3.xml, etc. these files have same format field , i created Foreach Loop container inside the container i have a dataflow which has xml source and Flat File Destination.
the destination should be a single file which combines data coming from the xml source instead i'm getting multiple outputs.

thanks
Andrew Watson-478275
Andrew Watson-478275
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2546 Visits: 2798
The only way I can see you getting multiple files from that connectionstring is if it was taking more than a second to process each file. Each time you go round the loop, it will rebuild the connection string, and will give it a different name because the seconds part will differ.

I would try building the filename into a new variable before entering the loop and then using that variable in the connectionstring.
Andrew Watson-478275
Andrew Watson-478275
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2546 Visits: 2798
(and obviously make sure you've got Overwrite=False in the Flat File Destination properties - I think that the default is True).
Nomvula
Nomvula
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1212 Visits: 714
i've build connection path variables -
ImportSourcePath = C:\OnKeyImports\WorkOrderCosting\Source\
FilePrefix = WorkOrderCosting
FileExtension = .txt

and build it in the connection string like this to add date in the file name:
RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) +
"_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) +
RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]
Andrew Watson-478275
Andrew Watson-478275
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2546 Visits: 2798
Yes, but it will recalculate that connectionstring every time it uses it (i.e. every time round the loop).

Create a new variable User::FullOutputFileName (or whatever), and assign its value as:

RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) + 
"_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) +
RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]


before entering the loop. (You'll probably need to do this in a script to ensure it gets the actual value rather than just the formula).
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)

Group: General Forum Members
Points: 86371 Visits: 21711
Andrew Watson-478275 (4/19/2013)
Yes, but it will recalculate that connectionstring every time it uses it (i.e. every time round the loop).

Create a new variable User::FullOutputFileName (or whatever), and assign its value as:

RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) + 
"_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) +
RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]


before entering the loop. (You'll probably need to do this in a script to ensure it gets the actual value rather than just the formula).


Another way of doing this is to have a static name for the output file and then to rename it after the data flows have all completed.


Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search