When you enable Service Broker you are enabling it for User processes. The End Points you set up are User end points. It is always enabled for System processes and the System end points are not shown to mere humans.

The same concept applies to CLR. Enabling CLR means enabling it for User processes. Many system processes use CLR and the use of CLR for these cannot be turned off or on. (OK, the SQL authors probably have a way of turning service Broker or CLR off for system processes but the rest of us do not.)