Can you copy the files to the remote server & then restore? Or restore successfully on Server A?

Sounds like a network problem rather than a SQL issue - can you or any network specialists check the NIC's & connections between the servers?