Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Dynamically load CSV files To Sql Server Tables


Dynamically load CSV files To Sql Server Tables

Author
Message
iamsql1
iamsql1
SSC Rookie
SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)

Group: General Forum Members
Points: 45 Visits: 139
Hi,
I am writing a SSIS package of copying CSV files to SQL tables.

Csv files are five in numbers in each multiple folders in Source Directory having same schema.

Problem is

how do i copy file to sql server table based on MATCHING NAMES of Files(.CSV) and Tables (of SQL).

Thanks,
Orlando Colamatteo
Orlando Colamatteo
SSCrazy Eights
SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)

Group: General Forum Members
Points: 8219 Visits: 14368
You can set the name of a source file in a Flat File Connection Manager and the table name in an OLE DB Destination dynamically using variables and Expressions. As long as the source csv file and destination tables are all the same schema it will work.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
iamsql1
iamsql1
SSC Rookie
SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)

Group: General Forum Members
Points: 45 Visits: 139
Thanks for your replay,,, yes i know the expression property of connection manager. Problem is :

i have files like EDGStock.csv , JHBStock.csv etc in Source PAth.

i have Stock Table in SQL Server.

how do i tell OLE DB connection manager that it should load only files having KEYWORD "STOCK" in every iteration of FOR LOOP Container.


Thanks
Orlando Colamatteo
Orlando Colamatteo
SSCrazy Eights
SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)

Group: General Forum Members
Points: 8219 Visits: 14368
You can decide that in the Expression of the Variable that determines which table name your OLE DB Destination points to.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
iamsql1
iamsql1
SSC Rookie
SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)

Group: General Forum Members
Points: 45 Visits: 139
can i use sql command for it?
Orlando Colamatteo
Orlando Colamatteo
SSCrazy Eights
SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)SSCrazy Eights (8.2K reputation)

Group: General Forum Members
Points: 8219 Visits: 14368
iamsql1 (2/1/2013)
can i use sql command for it?

No, Table name or view name variable.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
iamsql1
iamsql1
SSC Rookie
SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)

Group: General Forum Members
Points: 45 Visits: 139
HI
i have an idea. can i use a stored procedure and get it executed in EXECUTE SQL TASK

instead of using ole db destination for inserting , updating data in sql server db. ????
Phil Parkin
Phil Parkin
SSCrazy Eights
SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)

Group: General Forum Members
Points: 8300 Visits: 19447
iamsql1 (2/4/2013)
HI
i have an idea. can i use a stored procedure and get it executed in EXECUTE SQL TASK

instead of using ole db destination for inserting , updating data in sql server db. ????


Of course - but then the question becomes 'why bother with SSIS at all?'


Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
iamsql1
iamsql1
SSC Rookie
SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)

Group: General Forum Members
Points: 45 Visits: 139
Hi Phill
I am newbie to SSIS, i tried every effort to change the variable in olde db destination , but mapping of columns become issue.

i am stuck on it half a week., can u present specific example in it?
Phil Parkin
Phil Parkin
SSCrazy Eights
SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)SSCrazy Eights (8.3K reputation)

Group: General Forum Members
Points: 8300 Visits: 19447
iamsql1 (2/4/2013)
Hi Phill
I am newbie to SSIS, i tried every effort to change the variable in olde db destination , but mapping of columns become issue.

i am stuck on it half a week., can u present specific example in it?


OPC.Three made a critically important comment above:
As long as the source csv file and destination tables are all the same schema it will work.


My version of this is: "If column definitions in either the source or the destination are different, for any of the files, it will not work"

It seems that you have fallen foul of this.

You will need a separate data flow for every different source file format you have. And for every destination table whose columns are different (by name or by type).

It's difficult to be more specific without understanding the source and target schemas better.

But the important lesson to take away is the SSIS is largely driven by meta data which is generated at design time. Dynamic column mapping / naming / typing is not an easy option.


Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search