It's the elevated row estimations that will likely be the cause. The spools show up when the optimiser thinks it's better to cache intermediate results and read from the spool, rather than re-calculate.
They shouldn't be around for small row counts, as there's an overhead to write data to the spool.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability