Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Export large amount of data from one table to another Expand / Collapse
Author
Message
Posted Tuesday, October 1, 2013 2:09 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Wednesday, July 2, 2014 6:30 AM
Points: 253, Visits: 616
Hi Guys,

Is there a way to start up an export from one table to another, if the data export fails at some point? I want to create an SSIS package to load data to a table and in case the export fails at some point I can restart it again from the point that it fails.

Please advise.

Regards
IC
Post #1500294
Posted Tuesday, October 1, 2013 2:11 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Sunday, July 20, 2014 11:20 PM
Points: 13,252, Visits: 10,133
Normally no, because the load happens in a transaction.
It depends a bit on the settings of the OLE DB Destination. You might want to lower the batch size to try if batches are committed more quicker.

At a restart, what you would have to do is check the destination which rows are already loaded and exclude them from the source.

But, having smaller transactions/batches slows down the loading speed.




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1500296
Posted Tuesday, October 1, 2013 11:40 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Friday, June 27, 2014 2:05 PM
Points: 300, Visits: 810
I was thinking there was some support for checkpoints in ssis, tho I have never used them since we mostly do kill n fill.

My understanding is that you could restart from a checkpoint.
Post #1500515
Posted Tuesday, October 1, 2013 12:31 PM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Sunday, July 20, 2014 11:20 PM
Points: 13,252, Visits: 10,133
herladygeekedness (10/1/2013)
I was thinking there was some support for checkpoints in ssis, tho I have never used them since we mostly do kill n fill.

My understanding is that you could restart from a checkpoint.


Yes, but you cannot restart in the middle of a data flow.

Restart Packages by Using Checkpoints




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1500548
Posted Tuesday, October 1, 2013 1:45 PM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Friday, June 27, 2014 2:05 PM
Points: 300, Visits: 810
ah, yes, Makes sense to me. Possibly why I abandoned the idea for our use.

Post #1500577
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse