The accidental many-to-many join that Jeff mentioned is a possibility.  So is an implicit cast that required you to read an entire table is another, but there are others.  Take a look at your query plan and look for the yellow triangle icon on one of your steps.  Mouseover the icon and look for the warning near the bottom of the tooltip.