If you are using a non-covering index, then absolutely yes. The tipping point (where using a noncovering index and doing key lookups is less efficient than a table scan) is somewhere around 0.5% of the total rows in the table (it's ~ number of rows = 30% of the number of pages in the table)

There's a post on my blog "Seek or Scan" which shows this. Sorry, don't have time to find it, got to get to class.