November 13, 2008 at 8:57 pm
We have 2 sets of CSV flat files (37 columns and 15 columns). From the user perspective, these 2 type of files will treated as 1 file type. Then in the system somehow to need to cater this as seperate events.
So what we have done:
1. Connetion MAnager (flat file), we created 37 columns initially to cater this.
2. Some with Flat File Source as well OLE DB Destination with 37 columns.
What we found that if the data was loaded with 37 columns then this thing is ok BUT if we loaded with 15 columns data then the database fill in with almost 1/3 of this data due the data try to fill in column 16-37. So for instance we knew that from the original data that it has 12000 records but now in the datasbe almost 4000 records and we check the column 16-37 and there is a data in there.
The question are:
1. How to force only upload what ever in the original file is.
2. If there is no way to do this, how to detech the number of columns before proceeding this step.
3. Or am I better off to look at Bulk Insert instead.
I am appreciated your comment.
Thanks
November 14, 2008 at 7:32 am
You really need 2 data flow tasks which each have their own flat file connections, one that loads the 37 column files and one that loads the 15 column files.
Jack Corbett
Consultant - Straight Path Solutions
Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question
November 14, 2008 at 8:16 pm
Jack's idea is a good one and can, of course, be accomplished in a single package using an initial script task to check the number of columns with a conditional split based on the outcome.
Phil
Viewing 3 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply