You may have plan cache bloat. Glenn Alan Berry has some Diagnostic scripts for 2008 that can show you single use plans and lots of other stuff. I would check for those first, and decide how you want to proceed. If you have lots of dynamic sql coming in (from ORM possibly??), you can deal with that with a variety of approaches such as optimize for ad hoc workloads setting or even dbcc freeproccache. I have clients I had to set up jobs to do that on sql 2005 as frequently as every 30 minutes to aleviate memory pressure from single user ad hoc plans.
Kevin G. Boles
SQL Server Consultant
SQL MVP 2007-2012
TheSQLGuru on googles mail service