Need to process different file types in SSIS

Question

Need to process different file types in SSIS

apache626

SSChasing Mays

Points: 618
More actions
September 26, 2012 at 7:14 am

#264913

I have what I think will be a simple question. I need to have SSIS loop through a folder for only xls or txt files. I then need to base the processing on what type of file it is. I am trying to think of an approach. Should I move the files to separate folder based on their type and then process? Or should I have two different foreach loops to process the file types? Any help is appreciated!

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply

Ant-Green SSC Guru Points: 113445 More actions · Answer 1

Swings and roundabouts on that one, two folders one for xls the other for txt or two loops one for xls the other for txt will provide exactly the same methodoligy.

But then you have to look at if there is a process between them, say for example the xls is header information and the txt is the data and they need to be imported xls, txt, txt, txt, txt etc if so then two loops wont cut it as they will always insert in parallel or serial depending how you setup the precendence constraints etc.

Chandra Sekhara Vyas Dhara SSChampion Points: 10207 More actions · Answer 2

You better maintain two for each loop so the process will be faster

apache626 SSChasing Mays Points: 618 More actions · Answer 3

The file actually contains the same information. The excel file is manual entries. So the information will go into the same table.

apache626 SSChasing Mays Points: 618 More actions · Answer 4

What makes the 2 foreach loop faster in processing that the other way?

PChiragS SSCarpal Tunnel Points: 4965 More actions · Answer 5

you shoud use two for each loop to get each type of file from the same folder.

it makes the system fast, since you do not have to define the logic to get the files from the folder in single for each loop.

CGSJohnson SSCertifiable Points: 7208 More actions · Answer 6

You can keep them in the same folder. Within your foreach loop, however, inspect the file extension and have two branches off of the inspection object and do the separate processing in those branches.

Hope that this helps. Thanks.

- Chris

apache626 SSChasing Mays Points: 618 More actions · Answer 7

Thanks Chris and everyone for your responses. I have implemented a for each loop with a dummy script task to be able to set a constraint to check for file type and branch processing based on that. Thanks for all your help.