• I ran into another issue Tuesday and finally nailed it down to a file-copy operation that occurs prior to restore, but after the the block-level copy operation. I am just happy the block-level copy operation proved to be solid in these two occurrences as that is where the heavy lifting is done in terms of moving these large files between two geo-separated data centers. I added a piece of code to the process that later copies the files around in the remote data center to check the MD5 hash of the file before and after the problematic file-copy operation and to send an email if they do not match.

    So far I have not seen the issue again, but at least now I am confident I will be able to troubleshoot the issue fast if it happens in the future, hopefully before any users are affected. If this becomes a regular occurrence I am considering bringing ESEUtil into the mix. Per this article it seems to be quite reliable for copying large files over slow or poor network connections...much more so than what the script is doing, a CmdShell COPY operation:

    How to: Copy very large files across a slow or unreliable network by Grant Holliday

    There are no special teachers of virtue, because virtue is taught by the whole community.
    --Plato