Home Forums Data Warehousing Integration Services SSIS 2008 – Read roughly 50 CSV files from a folder, create SQL table from them dynamically, and dump data. RE: SSIS 2008 – Read roughly 50 CSV files from a folder, create SQL table from them dynamically, and dump data.

  • ankithimmatlalshah (4/21/2015)


    Hello Phil,

    Thank you for the quick response.

    For first point, I will re-confirm with customer that can we create the SQL script task that will first create the all tables to destination and then for each loop to migrate data from CSV file to SQL table?

    On second point, the tables will be created for very first time if it does not present, on next go (runs daily basis) it will append data roughly for 48 files while for 2 files it will purge some existing data, if present and then append the new data. The criteria for purge is yet to discussed with us.

    On third point, by same file name I meant that the same file name for a particular CVS file out of 50 with incremental number, like if I do have history.CSV then the complete data will be divided in 10 different files, like history_01.csv, history_02.csv files as so on. Sorry for the confusion.

    Please feel free to ask if any of the points are still not clear.

    OK, things are becoming clearer.

    First point ... agreed, though 'migrate' is probably better referred to as 'import'

    Second: for the two special files, I would suggest calling a stored proc (via ExecuteSQL task) after doing the load which performs the purge.

    Third: a foreach loop should take care of all of the files.

    If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.