You could use Office Automation although that is frowned upon for a service app. Not really a good idea.

It is always an issue with semi structured data and getting it...

Exactly that - Data Sources are templates to create connection managers. I never use them as I see no real gain.

You best be is to use configuration files. (Or SQL...

The same still applies barring the fact that you might as well setup your text file source to parse the first 3 columns and use a ragged right for the...

Sorry, should have mentioned to put a sleep in the loop. Sleep for n minutes / seconds before checking again. It will cause the same overhead as a periodically running...

You hit the nail on the head. What the salesman tells the bean counters and what is reality are two different things.

We had a rapidly growing, around 40-50GB a month,...

I think you should look at it again. It is really powerful and adaptable. I have yet to find a situation where it cannot get at the data reliably.

Post a...

if it is not comma delimited, what is it? You need something to split the columns. You could then just replace commas with an empty string.

While you could use a script component, I would say it's a bit of overkill for you.

You could use a flat file source, specify ragged right and create two columns....

Welcome to my world

There is actually a brilliant tool called Data Defractor which allows you to drill into the data. You specify boundries...

You could have an endless loop? Loop while 1<2 and check for the record. When the record exists, set the condition to be false so the loop ends. Also put...

I'm guessing using it with a normal client such as filezilla works fine?

On the ftp server, can you see open connections?

I would hazard a guess that it's an issue with...

This is an annoyance.

Your best bet is to enable Work Offline. Menu > SSIS > Work Offline.

The problem with this is you have to wait for all the connections to...

You don't necessarily need to init variables although I do and set them to stupid values like "Crispin Rocks". Yes, you do this in the design time variables pane.

If your...

hmmm, cannot see anything wrong with it.

If you put a watch on the variable, can you see the value change after you have "apparently" assigned it?

Put a break point on...

Step one: Create Data Flow

Step two: Drop XML Source component.

Step Three: Select XML file.

Step four: Click Create XSD

Step five: Read BOL Examples.