Thanks Grant. Using the transaction id (from sys.dm_tran_locks.request_owner_id) I could figure out that the same transaction also had an IX lock on an internal table called "queue_messages_180220113", which is likely why service broker could not clean up any more closed conversations. What was keeping me from getting any more info was that in sys.dm_tran_session_transaction no line existed for the transaction id, so I could not determine what process was holding the lock. My assumption is that it was either an internal process or something orphaned.

Any way, I have now restarted the sql server and automatically the closed conversations are now cleaned again plus I could finally end the one conversation that was in 'ER' state. i.e. the problem is gone now, leaving me with a little unsatisfied feeling that I couldn't figure out what was going on. Thanks for your response, though.