Questions from a newbie that's only been scripting queries for 2+ years:
You have it set up for deltas that are in table 2 that are not in table 1, to which data from table 2 is added to table 1, which is a great idea. I see that potential on the ETL that we do. My question on this issue is what happens to the deltas that are in table 1 that are not in table 2? What we do is an update on a personnel system here and some people get deleted from the system. So I wonder if this system can remove the ones that are removed when the data is updated. I have not studied this series in super detail, so if the answer is in your series, please let me know and I apologize for the inconvenience.
Related to above, we get text files that are imported into tables within a staging database. Could this work without the use of creating additional tables for a pre_load? If not, then I guess we can do a pre_stage table and use this process to increment our actual staging tables.
Someone suggested I explore this process for a possible alternative to using bcp queryout and/or Data Flow tasks for a bunch of files created from sql scripts. Would this be part of your Stairway discussion down the road?