Not necessarily because those pages read may be tossed out of cache fast. You can assume that a query with high physical reads is reading lots off the disk and may potentially be displacing lots of pages from cache.

Bear in mind though, that SQL intentionally and by design will grow its memory to the max it's allowed. It's called 'buffer pool rampup', so don't go looking for queries that are behaving correctly assuming that they are a problem. If you set max server memory to default, you're telling SQL Server that it can and should use all the memory on the server, so when it does use all the memory on the server, it's not a problem, it's doing what you told it that it could do.