Thanks all for the suggestions. What worked in the end was comparing the mssubscriptions table with a known good configuration and then manually adding in a record for the orphaned subscription.

Once there, the subscription could be successfully deleted.