You mention application queries.

This might imply you have other processes (IIS, SSAS, SSRS, etc.) running on the machiine.

These are not included in the SQL instance memory, even thought some may be part of the SQL install.

I'd start by looking in perfmon for indications that the system in paging out.

Process Explorer (part of the old Sys Internals toolset) might give a better look than task manager.

SQL Server only uses the memory it needs, and Max Memory is used as a guideline for it to clear out older / less frequently used items. Reducing this, and your results showing that very little really changed, shows that something else besides SQL Server is demanding memory.

I agree with Gail - this is what you need to track down.

You do not mention performance issues, just memory allocation.

Although I would suspect that performance be degraded started the hunt.

There are many things that can contribute to this, but start with the simple checks on what is really running.