Just some random thoughts:
Check the disks for contention
Check to see if anything else is running on the box and consuming inordinate CPU and/or memory
Make sure your anti-virus software isn't on and attempting to scan your database files
check the sql server for blocked processes
Depending on how far the data is travelling, you might have to have your network folks take a look at the traffic between source and mirror
And then again, I might be wrong ...