... I woud think that if there was a major problem with the query that it would always take a while to retrieve but that is not the case.
My question is if my queries actually do come back instantly, do I even spend any more time trying to tweak them or just write it off to a server, other issue and look elsewhere?
Actually, it can be quite the opposite: you may have real performance problem with your query, but sometimes it runs very faster. It can be due to caching.
Until you provide your queries, DDL's for tables involved (together with indexes) and current query execution plan, you very unlikely to get any relevant help from anyone here.