This same thing has happened to me and what resolved it in my case was to open up the MSreplication_agent_status table in tempdb and delete a record that had a publication named the same as the one I had been trying to get rid of in the first place. No more red Xs!