Take a look at the execution plan. It's possible that you're seeing scans across all those other tables. Maybe some indexes are in order.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning