Mystery solved:

Our Network Security group was running a scan that attempted to connect using the port SQL uses for database mirroring. Since the connect request was not a valid mirroring connection, errors were generated. This explains how the error also occurred on instances with no mirrored databases.

I'm glad I didn't follow one piece of advice found online and restart the mirror endpoints. Nothing broken on the SQL side. 😛