Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

multiple files with some columns are same and some columns are different but i have to load all this columns into single columns how Expand / Collapse
Author
Message
Posted Friday, February 7, 2014 2:39 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Wednesday, August 6, 2014 5:09 AM
Points: 9, Visits: 21
multiple files with some columns are same and some columns are different but i have to load all this columns into single table which is having all columns how is it possible with ssis 2008 r2 by using single data flow
Post #1539055
Posted Friday, February 7, 2014 2:55 AM
SSC-Addicted

SSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-Addicted

Group: General Forum Members
Last Login: Tuesday, September 16, 2014 2:39 AM
Points: 470, Visits: 483
Can you please give some additional details like file structure etc ? If possible please sample post the i/p file data and the output that you are expecting ...
Post #1539057
Posted Friday, February 7, 2014 3:15 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 1:10 PM
Points: 5,103, Visits: 11,906
..load all this columns into single columns ...


Please explain what you mean by this.

In general, for every different file format, you will need a separate dataflow. Multiple files with the same format can use the same dataflow, often by using a FOREACH container to loop round those files.



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

When you ask a question (and please do ask a question: "My T-SQL does not work" just doesn't cut it), please provide enough information for us to understand its context.
Post #1539065
Posted Friday, February 7, 2014 4:04 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Wednesday, August 6, 2014 5:09 AM
Points: 9, Visits: 21
data in file1 like this id,name,add and in file2 like this id,name,add,unit,sold and file3 is id,nam...,but i have all columns in one single destination table.and i want load this all data into my single destination.
Post #1539084
Posted Sunday, February 9, 2014 7:53 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 1:10 PM
Points: 5,103, Visits: 11,906
How many files are we talking about and also how many different layouts?

Multiple data flows (one per file type) is one way of handling this.

Another is to send all of the data to a script component and use that to handle the layout variations, in a single dataflow. In this case, your base file connection should define the entire row as a single string, to avoid any meta data issues between files.

Use a FOREACH container to iterate round the files (your dataflow will go in here).

If your C# is decent, the scripted option is likely to be tidier, in my opinion.



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

When you ask a question (and please do ask a question: "My T-SQL does not work" just doesn't cut it), please provide enough information for us to understand its context.
Post #1539564
Posted Sunday, February 9, 2014 7:11 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: General Forum Members
Last Login: Today @ 5:14 PM
Points: 35,266, Visits: 31,756
On the other hand, you could just use BULK INSERT pointed at different views that all point to the same table.

--Jeff Moden
"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

(play on words) "Just because you CAN do something in T-SQL, doesn't mean you SHOULDN'T." --22 Aug 2013

Helpful Links:
How to post code problems
How to post performance problems
Post #1539615
Posted Sunday, February 9, 2014 9:32 PM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Wednesday, August 6, 2014 5:09 AM
Points: 9, Visits: 21
Hi,
Actually i have 10 files which have 30 columns common in all files,but some files having additional colums that may be 10 columns only ,but i have this 40 columns in my destination table so i want to load this all files in single package.And if it is possible with script component please send code.i think it is clear now ..common colums in all files like this id,dealer name,addderes,............. and additional columns in file 1 is agent,sales no,and additional column in file 2 is product,..and in final destination table columns are id,dealer name,adderess,agent,sales no,product,(this table is having all columns).
Post #1539622
Posted Monday, February 10, 2014 12:29 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 1:10 PM
Points: 5,103, Visits: 11,906
And if it is possible with script component please send code ...


It is possible, but I'm not going to write the code for you - unless you want to hire me as a consultant.

Please take the time to do some research and have a go yourself. If you have specific implementation problems, feel free to post back and we will try to help.

I can help with pseudo-code though:

Foreach File
Determine row format
Foreach row
Break row into columns based on row format
Assign columns to output columns
Next row
Next File




Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

When you ask a question (and please do ask a question: "My T-SQL does not work" just doesn't cut it), please provide enough information for us to understand its context.
Post #1539640
Posted Monday, February 10, 2014 12:31 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 1:10 PM
Points: 5,103, Visits: 11,906
The above assumes that:

a) You have created appropriate output columns for your script component
b) That the script component is running in synchronous mode (one row in = one row out)



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

When you ask a question (and please do ask a question: "My T-SQL does not work" just doesn't cut it), please provide enough information for us to understand its context.
Post #1539641
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse