• dan-572483 (3/25/2014)


    The Excel file contains data going back several years and I'm only looking for data entered in the past few weeks. (Honestly it is a personal project I'm using to teach myself SSIS using a spreadsheet of gasoline purchases I've been keeping since the 1990s. I find myself more motivated to practice with data that has personal meaning to me than generic leaning guide projects on AdventureWorks )

    My goal is to learn to design projects efficiantly, which means reading in only data that is needed (the past week or two using a SELECT MAX() variable assigned from the target DB) rather than all 10+ years of data. If I connect the Conditional Split to the Source connector I can do that? I've also found messages about exporting Excel to CSV before importing to SQL, but that seems more complex that it should be.

    I haven't done exactly what you are doing, but I do have some comments.

    1) As Aaron has already pointed out, Excel doesn't do data-typing. Regardless of how Excel displays data, it is either text or numeric. Dates (I think) are stored as the number of days since 1/1/1900 (or maybe 1/1/1899). Decimals represent the time component. So I would try WHERE XLDate > nnnn (you'll have to work out what this number is).

    2) If you want to be efficient, don't use Excel files as data sources. Use CSV files instead.

    3) I understand your wanting to filter your source data, but using a conditional split to direct unwanted rows to oblivion is fast - you should try it. Remember, once again, that Excel is not a database engine, so queries against it are not likely to be optimised.

    If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.