It sounds like you are trying to create the output file meta data (ie, the column names and data types) dynamically, is that correct?

This isn't really how SSIS works -...

I imagine that this would evoke a better response if it were posted in the BI forum, as it's not really SSIS-related.

You have a sub called Button303_Click with no obvious means by which it will ever be executed - where did this code come from, a Windows Forms VB.NET project, perhaps?

You...

Based on the sample data and code, I cannot see why it would not work for you, as you appear to be doing the right things.

Or, as you are in...

John meant can't, not 'can'!

The Input0_ProcessInputRow sub is called for every row in the input data.

As long as the input data is in the required order, this allows the LastColA variable to be populated...

What is "@SSIS" supposed to represent in this expression?

John Rowan's first response gives you a solution to this, it would seem.

You need only a...

Do all of your flat files have the same schema?

What is in column 35? This is a tough one to answer without knowing a bit more about your data ...

OK, now that I have spent a bit more time on your solution, I can see what it's doing. Apologies for my previous post, but I haven't seen a solution...

benyos (8/29/2009)

---

Your 15 second solution is not SQL related at all:

download microsoft logparser and run it on your data.

if, for example, your data is in CSV file, you can run...

Maybe the initial fetch is causing problems.

Can you loop/batch the whole thing to SELECT and process x thousand rows at a time?

Actually, this is not quite true. The answer to the...

🙂 exactly where I was going - hence my question about schemas ...