Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Multiple flat files to import into multiple tables (different formats)


Multiple flat files to import into multiple tables (different formats)

Author
Message
Sirisha-631510
Sirisha-631510
Forum Newbie
Forum Newbie (1 reputation)Forum Newbie (1 reputation)Forum Newbie (1 reputation)Forum Newbie (1 reputation)Forum Newbie (1 reputation)Forum Newbie (1 reputation)Forum Newbie (1 reputation)Forum Newbie (1 reputation)

Group: General Forum Members
Points: 1 Visits: 53
I have about 30 flat files each in different formats with different column names.

I need to import them creating new tables respectively. I am open to let it pick the datatypes.
I just need them to get into the database.

Bulk insert doesn't work for me because it inserts into the same table.
I would have to create a new connection for each file to create a table so I wasn't sure if SSIS for each container could do any good.

we get many files from clients throughout the year and so far they were scattered around the year so import wizard or SSIS dataflow task with OLEDB conn worked.

These days I am on a tight deadline with these files so i need a quicker and automated way to do it.

Even a batch file would work. Please help!

Thanks,
Sirisha
kumar1
kumar1
Ten Centuries
Ten Centuries (1K reputation)Ten Centuries (1K reputation)Ten Centuries (1K reputation)Ten Centuries (1K reputation)Ten Centuries (1K reputation)Ten Centuries (1K reputation)Ten Centuries (1K reputation)Ten Centuries (1K reputation)

Group: General Forum Members
Points: 1042 Visits: 495
Hello Sirisha,

Did you find the solution for this?

I m planning to do the same with my current requirement. But I am dealing with only 3 file formats Text, excel and ASC respectively. So far I am just using import wizard, But requested to automate this.
Note: I really don't need to run a bunch per day, The task will be once a week.
Can you guide me if you already has the solution?
Raunak Jhawar
Raunak Jhawar
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1249 Visits: 1944
When you say that you have different file formats and file structures, you can leverage the following to design a SSIS package:

  • Create Dynamic File Connection Manager for all files

  • Use a For Each Loop to enumerate the file system directory to pick up the file and load the file onto the tables

  • Include a cleanup task to remove the file after successful load operation


  • Regards/Raunak
    Now a member of Linkedin

    Please visit the all new Performance Point Forum
    Please visit the all new Data Mining and Business Analytics Forum
    Phil Parkin
    Phil Parkin
    SSChampion
    SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)SSChampion (10K reputation)

    Group: General Forum Members
    Points: 10896 Visits: 19859
    Raunak Jhawar (12/17/2013)
    When you say that you have different file formats and file structures, you can leverage the following to design a SSIS package:

  • Create Dynamic File Connection Manager for all files

  • Use a For Each Loop to enumerate the file system directory to pick up the file and load the file onto the tables

  • Include a cleanup task to remove the file after successful load operation




  • You are responding to a forum newbie and these are high-level ideas which require some elucidation, in my opinion.

    I am no newbie, but still don't have a clue how would create a "Dynamic File Connection Manager for all files" - what is this thing?

    You mention the use of a FEL, but do not mention data flows at all. Perhaps you have a single 'Dynamic Data Flow for all files' in mind - if so, I'd like to know more please.


    Help us to help you. For better, quicker and more-focused answers to your questions, consider following the advice in this link.

    If the answer to your question can be found with a brief Google search, please perform the search yourself, rather than expecting one of the SSC members to do it for you.

    Please surround any code or links you post with the appropriate IFCode formatting tags. It helps readability a lot.
    dharmendra.mittal
    dharmendra.mittal
    Grasshopper
    Grasshopper (13 reputation)Grasshopper (13 reputation)Grasshopper (13 reputation)Grasshopper (13 reputation)Grasshopper (13 reputation)Grasshopper (13 reputation)Grasshopper (13 reputation)Grasshopper (13 reputation)

    Group: General Forum Members
    Points: 13 Visits: 80
    Hello Sirisha,

    Did you find the solution for this?
    Please let me know the approach you used.

    Thanks in advance
    Go


    Permissions

    You can't post new topics.
    You can't post topic replies.
    You can't post new polls.
    You can't post replies to polls.
    You can't edit your own topics.
    You can't delete your own topics.
    You can't edit other topics.
    You can't delete other topics.
    You can't edit your own posts.
    You can't edit other posts.
    You can't delete your own posts.
    You can't delete other posts.
    You can't post events.
    You can't edit your own events.
    You can't edit other events.
    You can't delete your own events.
    You can't delete other events.
    You can't send private messages.
    You can't send emails.
    You can read topics.
    You can't vote in polls.
    You can't upload attachments.
    You can download attachments.
    You can't post HTML code.
    You can't edit HTML code.
    You can't post IFCode.
    You can't post JavaScript.
    You can post emoticons.
    You can't post or upload images.

    Select a forum

































































































































































    SQLServerCentral


    Search