You are pulling back 6GB of data. If you're doing this in a client, you might have rendering issues, as well as IO/memory issues on the server. 6GB of data will slide through the buffer cache and clear out things.

If any other user is querying the table, they might be blocking. Check sp_who2 for blocking connections. Don't do a "select *" unless you really need all that data.