to answer the question, there is a changelog table associated with each table that has the changedate and a calculated modification number, so the initial extract is all records in the changelog since the last successful refresh, but a quick filter is done on the modification number to throw out records that don't require updating. I'd like to get rid of the modification number and just filter on the changedate, but since I inherited the process, I'm moving slowly with any modifications.

This all started because some tables are moving to another server, which also changes the modification number in the changelogs (which did not move but were recreated), and is causing some heartburn trying to decide if I need to refresh the new changelog tables completely from the previous server to synch them up. (old server is going away, so..) or just synch last X months and go from there.

anyway, I've done some analysis of the times involved in the extracts, transforms, and delivery of new or updated data since I originally posted, and quite frankly, the extract phase where the changelog information plays is actually not my worse nightmare, but further into the process where some subqueries of subqueries lurk.