Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

SSIS iSeries Source Duplicates Records Expand / Collapse
Author
Message
Posted Monday, January 28, 2013 12:22 PM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, November 24, 2014 3:48 PM
Points: 38, Visits: 542
I am creating a package to read data from our production iSeries (AS/400) and copying it into our new data warehouse. I am reading millions of rows at a time, and a couple of them will be updated while I am doing the read. The SSIS package is picking these records up twice, once before, and once after the update. I have tried changing the isolation level, but no luck.

http://publib.boulder.ibm.com/infocenter/iseries/v5r4/index.jsp?topic=/db2/rbafzmstisol.htm
Looking at footnote P2, it appears that this may be by design. I have looked for a READPAST hint or something similar, but I haven't found anything.

Has anyone else dealt with this issue? Is there a way to solve it short of reading all the records into a pre-staging table prior to loading the staging table?
Post #1412601
Posted Thursday, September 11, 2014 1:51 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Thursday, October 23, 2014 7:32 AM
Points: 1, Visits: 9
I'm facing the same problem, which Isolation levels did you try? I have two options: ReadCommited and Snapshot. Currently it is on ReadCommited, before I'm changing it to Snapshot, I like to know if you already tested that?
Thanks
Post #1612484
Posted Monday, September 15, 2014 10:02 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, November 24, 2014 3:48 PM
Points: 38, Visits: 542
Sorry, I never found a way to handle this through the select statement. I just dropped the records into a staging table and then filtered out the duplicates.
Post #1613743
Posted Monday, September 15, 2014 10:29 AM
SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Today @ 11:11 AM
Points: 2,533, Visits: 7,096
f.welten (9/11/2014)
I'm facing the same problem, which Isolation levels did you try? I have two options: ReadCommited and Snapshot. Currently it is on ReadCommited, before I'm changing it to Snapshot, I like to know if you already tested that?
Thanks


Quick thought, snapshot should work fine, just make certain that tempdb is properly configured if you have large number of updates.
Post #1613750
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse