Task manager lies. Don't use task manager to check SQL Server's memory usage, it will often show far too low (limitation in Task Manager). Use performance monitor and the Total Server Memory counter.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability