SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


multiple files with some columns are same and some columns are different but i have to load all this...


multiple files with some columns are same and some columns are different but i have to load all this columns into single columns how

Author
Message
bayapareddy.msbi
bayapareddy.msbi
SSC Rookie
SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)

Group: General Forum Members
Points: 33 Visits: 21
multiple files with some columns are same and some columns are different but i have to load all this columns into single table which is having all columns how is it possible with ssis 2008 r2 by using single data flow
SQLCJ
SQLCJ
Say Hey Kid
Say Hey Kid (695 reputation)Say Hey Kid (695 reputation)Say Hey Kid (695 reputation)Say Hey Kid (695 reputation)Say Hey Kid (695 reputation)Say Hey Kid (695 reputation)Say Hey Kid (695 reputation)Say Hey Kid (695 reputation)

Group: General Forum Members
Points: 695 Visits: 576
Can you please give some additional details like file structure etc ? If possible please sample post the i/p file data and the output that you are expecting ...
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)

Group: General Forum Members
Points: 52926 Visits: 21195
..load all this columns into single columns ...


Please explain what you mean by this.

In general, for every different file format, you will need a separate dataflow. Multiple files with the same format can use the same dataflow, often by using a FOREACH container to loop round those files.


Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
bayapareddy.msbi
bayapareddy.msbi
SSC Rookie
SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)

Group: General Forum Members
Points: 33 Visits: 21
data in file1 like this id,name,add and in file2 like this id,name,add,unit,sold and file3 is id,nam...,but i have all columns in one single destination table.and i want load this all data into my single destination.
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)

Group: General Forum Members
Points: 52926 Visits: 21195
How many files are we talking about and also how many different layouts?

Multiple data flows (one per file type) is one way of handling this.

Another is to send all of the data to a script component and use that to handle the layout variations, in a single dataflow. In this case, your base file connection should define the entire row as a single string, to avoid any meta data issues between files.

Use a FOREACH container to iterate round the files (your dataflow will go in here).

If your C# is decent, the scripted option is likely to be tidier, in my opinion.


Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Jeff Moden
Jeff Moden
SSC Guru
SSC Guru (219K reputation)SSC Guru (219K reputation)SSC Guru (219K reputation)SSC Guru (219K reputation)SSC Guru (219K reputation)SSC Guru (219K reputation)SSC Guru (219K reputation)SSC Guru (219K reputation)

Group: General Forum Members
Points: 219622 Visits: 42002
On the other hand, you could just use BULK INSERT pointed at different views that all point to the same table.

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
If you think its expensive to hire a professional to do the job, wait until you hire an amateur. -- Red Adair

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
bayapareddy.msbi
bayapareddy.msbi
SSC Rookie
SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)SSC Rookie (33 reputation)

Group: General Forum Members
Points: 33 Visits: 21
Hi,
Actually i have 10 files which have 30 columns common in all files,but some files having additional colums that may be 10 columns only ,but i have this 40 columns in my destination table so i want to load this all files in single package.And if it is possible with script component please send code.i think it is clear now ..common colums in all files like this id,dealer name,addderes,............. and additional columns in file 1 is agent,sales no,and additional column in file 2 is product,..and in final destination table columns are id,dealer name,adderess,agent,sales no,product,(this table is having all columns).
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)

Group: General Forum Members
Points: 52926 Visits: 21195
And if it is possible with script component please send code ...


It is possible, but I'm not going to write the code for you - unless you want to hire me as a consultant.

Please take the time to do some research and have a go yourself. If you have specific implementation problems, feel free to post back and we will try to help.

I can help with pseudo-code though:

Foreach File
Determine row format
Foreach row
Break row into columns based on row format
Assign columns to output columns
Next row
Next File




Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Phil Parkin
Phil Parkin
SSC Guru
SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)SSC Guru (52K reputation)

Group: General Forum Members
Points: 52926 Visits: 21195
The above assumes that:

a) You have created appropriate output columns for your script component
b) That the script component is running in synchronous mode (one row in = one row out)


Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search