I suspect that all of this comes out of a desire to make varchar and char equality testing easy. In the case of char equality testing, it seems obvious that we don't want to consider the trailing spaces with which they are padded, so ANSI says to pad the shorter string out with spaces until it is the same length as the longer string, then check equality. In order to make it easy to compare a varchar and a char, the same logic is also applied. It's when it gets applied to comparing a varchar with a varchar that it becomes non-intuitive, but in the interest of consistency, even for varchars SQL defines '' = ' ' as TRUE.

With that in place, consider that the null byte is the binary equivalent of a space in a char field. The same logic then applies - we want comparison of binary values to be straightforward, and thus in the interest of consistency SQL pads varbinary values with null bytes. Thus any length of null bytes is equal to an empty string, and this has nothing to do with processor endianness, CPU data register widths, etc.