Don't use Task Manager to check SQL's memory, it lies. The labels on the values are also not as clear as they could be.
If SQL is still using too much memory, reduce max server memory slightly. Repeat until you're happy with the memory situation.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability