November 16, 2009 at 12:19 pm
Here's the situation I have -
I have a raw file which contains ID numbers. Those ID numbers correspond with a unique ID field of a table in my database.
What I want to do is to read in those ID numbers from the raw file, and then obtain other information related to them. This other information is located in specific tables.
I've done some looking around, and the only option I can find which seems to work is to use the Lookup transform, and pass the ID numbers through it to get the information related to the IDs.
Is this the only option? It seems like it's a bit excessive, since it will require me to load all of the information for all records into memory, when I have already got the IDs for which I need the information.
If I were writing this out in a T-SQL stored procedure, the way that I would be doing it would be to obtain the raw file data into a temp table, and then write a query along the lines of "SELECT FROM TABLE JOIN RAW_FILE ON TABLEID = RAW_FILEID". Can I do something similar in SSIS?
November 16, 2009 at 9:53 pm
You could limit the amount of memory it uses by setting up memory restrictions on the lookup.
CEWII
November 19, 2009 at 2:33 am
You could sort the ID's and use merge join transformation to do the lookup.
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply