BSavoie (1/23/2010)
Thanks Phil, this is really not about employees at all. I just thought that might make my example a little easier to explain. This is really about exporting data to a vendor once per month. Each month, I have to make sure I don't send anything that was sent the previous month, so I keep track of the MAX record id every month and use that as a starting point the next month.I'll give your example a try. So anything I put in the data flow gets executed once for each record. Do I understand that correctly?
I think you need to consider this a bit more. In your example, what happens if you send that data on Employee A last month, that employee is terminated this month - but, since it has already been sent you won't be sending it again?
In other words, what about updates to the system that need to be updated in the downstream systems? How are you going to identify those?
Find the column that identifies the last updated date for that entity (or creation date). Then use that date to filter for any records that have been modified since the last time you extracted the data.
Jeffrey Williams
“We are all faced with a series of great opportunities brilliantly disguised as impossible situations.”
― Charles R. Swindoll
How to post questions to get better answers faster
Managing Transaction Logs