Ok, so when I drill down in Windows Explorer to the physical file, the size of it there is not reflecting the amount of data in it (meaning it is full) but that it is that size because a allocated that much size to it in my sql server configuration.

So if I made my allocated file size for the tempdb 4.5 GB, that physical file, located at Drive:\Program Files\Microsoft SQL Server\MSSQL\Data\tempdb.mdf, is always going to be that size. Or bigger perhaps, but never smaller.