February 11, 2013 at 4:35 pm
Hello, I want to import all the rows from a source table where the value of a column (ID) is greater than max(ID) in the target table (thereby getting only new rows).
Lookup transforms do joins, and both tables are > 200M rows, so that seems like a waste when all I need is a where filter.
This sounds simple, but I cant figure it out using SSIS. Linked servers would solve this, but that's not an option here.
Thanks in advance for any help.
February 11, 2013 at 7:11 pm
I would do this in two steps:
1. Run an Execute SQL Task to retrieve the max ID from the destination table and store the result in a variable.
2. Run a Data Flow Task to copy the data from the source where the ID is greater than the ID you retrieved from step 1.
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
February 12, 2013 at 2:07 am
opc.three (2/11/2013)
I would do this in two steps:1. Run an Execute SQL Task to retrieve the max ID from the destination table and store the result in a variable.
2. Run a Data Flow Task to copy the data from the source where the ID is greater than the ID you retrieved from step 1.
+1 exactly how I would do it.
If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply