Are you receiving any Sort warnings? these can be a clue of ORDER BY spilling into TempDB and insufficient memory. Check Statistics are up to date