It seems the solution is to make sure the type is binary comparable and to set IsByteOrdered = true in the arguments to the Microsoft.SqlServer.Server.SqlUserDefinedType attribute.

Of course, the problem then is to ensure that the binary format is correctly ordered!