dan-572483 (3/25/2014)
The Excel file contains data going back several years and I'm only looking for data entered in the past few weeks. (Honestly it is a personal project I'm using to teach myself SSIS using a spreadsheet of gasoline purchases I've been keeping since the 1990s. I find myself more motivated to practice with data that has personal meaning to me than generic leaning guide projects on AdventureWorks )My goal is to learn to design projects efficiantly, which means reading in only data that is needed (the past week or two using a SELECT MAX() variable assigned from the target DB) rather than all 10+ years of data. If I connect the Conditional Split to the Source connector I can do that? I've also found messages about exporting Excel to CSV before importing to SQL, but that seems more complex that it should be.
If you connect the conditional split to the source, you can filter out rows but only right after they were read.
So you are still reading everything in. In my opinion, that's OK. Some types of input are not easy to filter (such as flat files) and it might be easier to just read everything. Excel files can contain only so much rows - depending on the version - so you have to ask yourself if it is worth investing a lot of development time in gaining just a few seconds of performance.
Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP