Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase 12»»

Dynamically load CSV files To Sql Server Tables Expand / Collapse
Author
Message
Posted Friday, February 1, 2013 6:11 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, September 16, 2013 11:20 AM
Points: 45, Visits: 139
Hi,
I am writing a SSIS package of copying CSV files to SQL tables.

Csv files are five in numbers in each multiple folders in Source Directory having same schema.

Problem is

how do i copy file to sql server table based on MATCHING NAMES of Files(.CSV) and Tables (of SQL).

Thanks,
Post #1414625
Posted Friday, February 1, 2013 6:53 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Yesterday @ 7:01 PM
Points: 7,139, Visits: 12,762
You can set the name of a source file in a Flat File Connection Manager and the table name in an OLE DB Destination dynamically using variables and Expressions. As long as the source csv file and destination tables are all the same schema it will work.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1414654
Posted Friday, February 1, 2013 11:35 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, September 16, 2013 11:20 AM
Points: 45, Visits: 139
Thanks for your replay,,, yes i know the expression property of connection manager. Problem is :

i have files like EDGStock.csv , JHBStock.csv etc in Source PAth.

i have Stock Table in SQL Server.

how do i tell OLE DB connection manager that it should load only files having KEYWORD "STOCK" in every iteration of FOR LOOP Container.


Thanks
Post #1414778
Posted Friday, February 1, 2013 12:43 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Yesterday @ 7:01 PM
Points: 7,139, Visits: 12,762
You can decide that in the Expression of the Variable that determines which table name your OLE DB Destination points to.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1414810
Posted Friday, February 1, 2013 2:49 PM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, September 16, 2013 11:20 AM
Points: 45, Visits: 139
can i use sql command for it?
Post #1414856
Posted Friday, February 1, 2013 3:09 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Yesterday @ 7:01 PM
Points: 7,139, Visits: 12,762
iamsql1 (2/1/2013)
can i use sql command for it?

No, Table name or view name variable.


__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1414862
Posted Monday, February 4, 2013 1:47 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, September 16, 2013 11:20 AM
Points: 45, Visits: 139
HI
i have an idea. can i use a stored procedure and get it executed in EXECUTE SQL TASK

instead of using ole db destination for inserting , updating data in sql server db. ????
Post #1415166
Posted Monday, February 4, 2013 2:10 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 3:03 AM
Points: 5,314, Visits: 12,341
iamsql1 (2/4/2013)
HI
i have an idea. can i use a stored procedure and get it executed in EXECUTE SQL TASK

instead of using ole db destination for inserting , updating data in sql server db. ????


Of course - but then the question becomes 'why bother with SSIS at all?'



Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

When you ask a question (and please do ask a question: "My T-SQL does not work" just doesn't cut it), please provide enough information for us to understand its context.

It is better to keep your mouth shut and appear stupid than to open it and remove all doubt. (Mark Twain)
Post #1415184
Posted Monday, February 4, 2013 3:52 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, September 16, 2013 11:20 AM
Points: 45, Visits: 139
Hi Phill
I am newbie to SSIS, i tried every effort to change the variable in olde db destination , but mapping of columns become issue.

i am stuck on it half a week., can u present specific example in it?
Post #1415211
Posted Monday, February 4, 2013 6:31 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 3:03 AM
Points: 5,314, Visits: 12,341
iamsql1 (2/4/2013)
Hi Phill
I am newbie to SSIS, i tried every effort to change the variable in olde db destination , but mapping of columns become issue.

i am stuck on it half a week., can u present specific example in it?


OPC.Three made a critically important comment above:
As long as the source csv file and destination tables are all the same schema it will work.


My version of this is: "If column definitions in either the source or the destination are different, for any of the files, it will not work"

It seems that you have fallen foul of this.

You will need a separate data flow for every different source file format you have. And for every destination table whose columns are different (by name or by type).

It's difficult to be more specific without understanding the source and target schemas better.

But the important lesson to take away is the SSIS is largely driven by meta data which is generated at design time. Dynamic column mapping / naming / typing is not an easy option.




Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

When you ask a question (and please do ask a question: "My T-SQL does not work" just doesn't cut it), please provide enough information for us to understand its context.

It is better to keep your mouth shut and appear stupid than to open it and remove all doubt. (Mark Twain)
Post #1415265
« Prev Topic | Next Topic »

Add to briefcase 12»»

Permissions Expand / Collapse