As Gail indicated, SQL Server will not release memory unless the OS is under memory pressure and forces SQL Server to releasse memory.

As she also said, you may want to reduce the MAX MEMORY setting from 6 GB to 4 or 5 GB and see what occurs.

Do not expect SQL Server to dynamically allocate and release memory. When first started, it may not take much, but as SQL Server caches data, query plans, etc, it will continue to use more memory until it reaches your max setting. Even then, it may still use some additional memory depending on what other components you are using, such as CLR ( as pointed out by Gail as well).