Post reply

Load .csv files with "almost" identical structure through SSIS in a simple way

  • December 7, 2011 at 8:56 pm

    #251762

    Hi guys,

    I have the following problem that I need help on:

    I am given a set of CSV files which I need to load through SSIS into the database table. The files are comma delimited with double quotes as a text qualifier and have an almost identical structure with the following difference - some of the files have a comma at the end of each row while other files do not have the comma at the end of each row.

    Example:

    First set of CSV Files have the following structure:

    "abc", "def", "ghi",

    "jkl", "mnop", "qrst",

    Second set of CSV Files have the following structure:

    "abc", "def", "ghi"

    "jkl", "mnop", "qrst"

    I want to have all of the files together in 1 folder and process them together.

    What is the easiest way to do this?

    Would I need to perform some preprocessing using some script task, or is there a simpler way to deal with these "almost" identical csv file structures in a generic way, and process them together?

    Please advise.

    Thank you!

  • December 7, 2011 at 11:25 pm

    #1418478

    Although they almost look identical, they are surely not. The first one has 4 column, the last one 3.

    You can do some preprocessing, by removing the final comma from the first file and then process all types of files with the same dataflow. Or you can read all files as one column, and split it yourself using a script component.

    Need an answer? No, you need a question
    My blog at https://sqlkover.com.
    MCSE Business Intelligence - Microsoft Data Platform MVP

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply