Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Avoid Loading the same Excel File twice to a OLE DB Destination Expand / Collapse
Author
Message
Posted Saturday, April 26, 2014 7:02 PM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Monday, May 12, 2014 3:44 PM
Points: 20, Visits: 65
Hi Everyone,
I have an SSIS package that loops through a folder looking for and then loading excel files into SQL Server.

Does anyone have a method to stop SSIS from loading the same excel file twice.

Kind Regards
Post #1565382
Posted Sunday, April 27, 2014 3:14 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 10:41 AM
Points: 7,040, Visits: 12,954
What do you mean by "loading twice"?

Does the current loop perform the "double load" or does it happen when the SSIS package is restarted?

There are several options to avoid the "reload":
The easiest is to transfer the file to a different folder once it is processed.
Another option would be to store the last execution time of the import task and load only files with a more recent creation or modification date.
Yet another solution could be a table of files already imported and check against that table during import.




Lutz
A pessimist is an optimist with experience.

How to get fast answers to your question
How to post performance related questions
Links for Tally Table , Cross Tabs and Dynamic Cross Tabs , Delimited Split Function
Post #1565403
Posted Sunday, April 27, 2014 6:30 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Today @ 10:35 AM
Points: 1,292, Visits: 3,711
LutzM (4/27/2014)

There are several options to avoid the "reload":
The easiest is to transfer the file to a different folder once it is processed.
Another option would be to store the last execution time of the import task and load only files with a more recent creation or modification date.
Yet another solution could be a table of files already imported and check against that table during import.


A hash / fingerprint such as MD5 would guard against a file being resubmitted with a different name or dates.
Post #1565413
Posted Sunday, April 27, 2014 3:38 PM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Monday, May 12, 2014 3:44 PM
Points: 20, Visits: 65
Thanks for all your help. I will try your suggestion.
Post #1565430
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse