What's infeasible is performing scalar operations on it:
Address1 = "12 West Broadway, New York, NY 10036"
Address2 = "941 Market Street, San Francisco, CA 94109"
Address3 = Address1 + Address2
Address4 = Address1 - Address2
IF Address1 < Address2 THEN ...
As you pointed out, I don't see any advantage to a DistanceFrom "operator" over creating it as a UDF.