April 10, 2014 at 8:28 am
Hello All,
I have files which has date in file name and I want to load all files in sequential order.
Here I am giving you an example of my file name.
"NetworkActivity_869_403722_01-01-2014.log"
"NetworkActivity_869_403722_01-02-2014.log"
"NetworkActivity_869_403722_01-03-2014.log"
"NetworkActivity_869_403722_01-04-2014.log"
"NetworkActivity_869_403722_01-05-2014.log"
These are my files in that I want to load files sequentially means jan 1st, 2nd like this way.
Any suggestions?
April 10, 2014 at 8:35 am
What are you using to load the files? BULK INSERT, bcp, SSIS, other?
John
April 10, 2014 at 8:40 am
I am using Flat File connection as a source in SSIS.
April 10, 2014 at 8:41 am
I mean I am using foreach loop container in that I have Flat File Connection as a source and PDW destination.
April 10, 2014 at 8:52 am
As far as I know, there's not any way to specify the order in which a ForEach Loop processes. I think you'd need to use a Script Task to guarantee the order. Another alternative is to load everything in the order it comes, but add a derived column with the name of the file. You can then use that column to process the imported data in the order of the file it came from.
John
April 10, 2014 at 8:55 am
You can do it in different ways.
1) Have a table with all file names and using for each loop get the file name in an order and use it.
2) If the files are not in a order in the location, Use expression builder to build the expression every time to create the file by incrementing the counter, so that date increments by one after each successful file load.
3) If the files are rightly ordered in the source as per file name, they will follow the same order.
I think these would be enough for now.
Good Luck 🙂 .. Visit www.sqlsaga.com for more t-sql code snippets and BI related how to articles.
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply