Post reply

Problems with incremental load

  • September 10, 2008 at 9:25 am

    #124559

    I followed Andy's method for incremental loads

    http://www.sqlservercentral.com/articles/SSIS/62063/

    A very good explanation of how to do incremental loads. I have implemented it and had no problems in the beginning. But recently i am having some issues and wondering if anybody could help me out.

    Previuosly me incremental data was less ( around 3000 rows) and it worked well. But now I have about 12000 records with about 9000 for insert and about 3500 for update. The problem I have is that when i run the package it does not come out of the debug mode and hangs there. If i look into the data flow task, it shows the total no:of rows, no:of insert and update rows but the packages just freezes there. i have to then manually stop the package.

    I looked up online and read somewhere that I was due to cache size in lookup transformation. I changed its properties from full cache to none. Now sometimes the data gets loaded and sometimes it doesnt. If I break my source file into smaller sets and then do the load, it works. Is it any solution for this and what might be the reason for this behaviour?

  • September 10, 2008 at 1:55 pm

    #869957

    hey can you just post a screen shot of the data flow task. I think that should help us.

    and when you are running the package go to the last tab which gives the execution information and check if some information is posted over there about there.It can be a buffer swapping issue or probably the look up is taking time to cache. So check out there if any information is being provided by DTS.

    Regards
    Venkat
    http://sqlblogging.blogspot.com[/url]

  • September 10, 2008 at 1:57 pm

    #869958

    One more option is try to reduce the buffer size in the package.

    Regards
    Venkat
    http://sqlblogging.blogspot.com[/url]

  • September 10, 2008 at 11:26 pm

    #870112

    Hi Sonia..you can also check the size of your tempDB that may also cause you a problem and along with that check the table lock also.

  • September 10, 2008 at 11:45 pm

    #870114

    the screenshot of the package had been attached. i hope this provides some more insight into the situation we are looking into

  • September 11, 2008 at 2:04 pm

    #870623

    i was finally able to overcome the problem. previuosly i was following the method in the link below to update my rows

    http://vsteamsystemcentral.com/cs21/blogs/applied_business_intelligence/archive/2007/05/21/ssis-design-pattern-incremental-loads.aspx

    and i was having problems. but now i am implementing it exactly like the way Mr.Andy tells as mentioned in the below article

    http://vsteamsystemcentral.com/articles/IncrementalLoadUPDATED.doc

    and it works absolutely. still cant figure out why the previous one was not working

Viewing 6 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic. Login to reply