You may have to delete the mirroring endpoint as well if there are no databases mirrored. There are various other internal tasks to SQL Server that use Service Broker also. For details on Service Broker waits see the blog below.

http://blogs.msdn.com/b/sql_service_broker/archive/2008/12/01/service-broker-wait-types.aspx