At least not completely. There are some functions in SQL Server that are built on the SQL CLR, and even if you have disabled the CLR for user written code, it still loads app domains and functions for system code.
An example of this is the spatial data type. It’s a SQL CLR written user defined data type, but written by Microsoft. If you create a spatial variable, the SQL CLR loads an app domain and code for the UDT using spatial data.
Not a big problem, and not a security risk from my standpoint, but it is worth knowing if you have to debug into memory or some detailed traces/events and see the SQL CLR loading. Don’t panic, and understand that you can’t prevent this from loading for these system objects.