Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase 12»»

load multiple xml file into a single flat file destination Expand / Collapse
Author
Message
Posted Thursday, April 18, 2013 5:31 AM
SSC-Enthusiastic

SSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-Enthusiastic

Group: General Forum Members
Last Login: Friday, December 20, 2013 4:53 AM
Points: 119, Visits: 411
hi guys
i need help, i've created a ForeachLoop Container which loop through all the files .xml in my source directory then created XML Source and flat file Destination, my package execute successfully but create multiple output .txt file instead i one to output in one single .txt file

thanks
Post #1443752
Posted Friday, April 19, 2013 4:57 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: 2 days ago @ 2:21 AM
Points: 1,170, Visits: 2,153
What is the connectionstring set to in the connection manager for the output file? I can't see how you can get multiple files unless you're renaming dynamically during the loop.
Post #1444308
Posted Friday, April 19, 2013 5:04 AM
SSC-Enthusiastic

SSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-Enthusiastic

Group: General Forum Members
Last Login: Friday, December 20, 2013 4:53 AM
Points: 119, Visits: 411
thank you for the response
here's my output path C:\OnKeyImports\WorkOrderCosting\Source\WorkOrderCosting_20130419_100035.txt

the expression on the connection string is @[User::ImportSourcePath] + @[User::FilePrefix] + "_" + (DT_WSTR,4)DatePart("yyyy", GetDate()) + RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) + "_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]
Post #1444310
Posted Friday, April 19, 2013 5:08 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: 2 days ago @ 2:21 AM
Points: 1,170, Visits: 2,153
Are you getting multiple files with different times in the name, or are you getting a single file which is being overwritten each time round the loop?
Post #1444311
Posted Friday, April 19, 2013 5:23 AM
SSC-Enthusiastic

SSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-Enthusiastic

Group: General Forum Members
Last Login: Friday, December 20, 2013 4:53 AM
Points: 119, Visits: 411
i'm getting multiple files outputs whereas i want to combine all my input to a single file.

here's what i'm doing. i have multiple files e.g Item1.xml, Item2.xml, Item3.xml, etc. these files have same format field , i created Foreach Loop container inside the container i have a dataflow which has xml source and Flat File Destination.
the destination should be a single file which combines data coming from the xml source instead i'm getting multiple outputs.

thanks
Post #1444317
Posted Friday, April 19, 2013 5:33 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: 2 days ago @ 2:21 AM
Points: 1,170, Visits: 2,153
The only way I can see you getting multiple files from that connectionstring is if it was taking more than a second to process each file. Each time you go round the loop, it will rebuild the connection string, and will give it a different name because the seconds part will differ.

I would try building the filename into a new variable before entering the loop and then using that variable in the connectionstring.
Post #1444323
Posted Friday, April 19, 2013 5:39 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: 2 days ago @ 2:21 AM
Points: 1,170, Visits: 2,153
(and obviously make sure you've got Overwrite=False in the Flat File Destination properties - I think that the default is True).
Post #1444328
Posted Friday, April 19, 2013 5:45 AM
SSC-Enthusiastic

SSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-Enthusiastic

Group: General Forum Members
Last Login: Friday, December 20, 2013 4:53 AM
Points: 119, Visits: 411
i've build connection path variables -
ImportSourcePath = C:\OnKeyImports\WorkOrderCosting\Source\
FilePrefix = WorkOrderCosting
FileExtension = .txt

and build it in the connection string like this to add date in the file name:
RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) +
"_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) +
RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]
Post #1444331
Posted Friday, April 19, 2013 5:56 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: 2 days ago @ 2:21 AM
Points: 1,170, Visits: 2,153
Yes, but it will recalculate that connectionstring every time it uses it (i.e. every time round the loop).

Create a new variable User::FullOutputFileName (or whatever), and assign its value as:

RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) + 
"_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) +
RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]


before entering the loop. (You'll probably need to do this in a script to ensure it gets the actual value rather than just the formula).
Post #1444334
Posted Friday, April 19, 2013 6:03 AM


SSCarpal Tunnel

SSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal Tunnel

Group: General Forum Members
Last Login: Today @ 12:24 PM
Points: 4,828, Visits: 11,183
Andrew Watson-478275 (4/19/2013)
Yes, but it will recalculate that connectionstring every time it uses it (i.e. every time round the loop).

Create a new variable User::FullOutputFileName (or whatever), and assign its value as:

RIGHT("0" + (DT_WSTR,2)DatePart("mm", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("dd", GetDate()), 2) + 
"_" + RIGHT("0" + (DT_WSTR,2)DatePart("hh", GetDate()), 2) + RIGHT("0" + (DT_WSTR,2)DatePart("mi", GetDate()), 2) +
RIGHT("0" + (DT_WSTR,2)DatePart("ss", GetDate()), 2) + @[User::FileExtension]


before entering the loop. (You'll probably need to do this in a script to ensure it gets the actual value rather than just the formula).


Another way of doing this is to have a static name for the output file and then to rename it after the data flows have all completed.



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

When you ask a question (and please do ask a question: "My T-SQL does not work" just doesn't cut it), please provide enough information for us to understand its context.
Post #1444335
« Prev Topic | Next Topic »

Add to briefcase 12»»

Permissions Expand / Collapse