Don't use Task Manager to check SQL Server's memory usage. In many cases it will report a value far, far, far lower than what SQL is actually using. Use Perfmon and check the Total Server Memory counter.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability