• Pete,

    No, by approximate, it is not meant that no finite numbers are representable, for example 1, .9, .1, and .005 are representable just fine. It is just an acknowledgment that certain numbers are not representable in base two, just as some numbers are not representable in base 10

    In the example given, the problem is that .3 and .7 are not representable. A simple algorithm for converting a base 10 fraction to a base 2 fraction is as follows:

    set number = .1 (remember, this is binary, .1 = 2^-1, or 1/(2^1) or .5 in decimal)

    set exp = -1

    while number <> target

    if number > target

    set number = number - 2^exp

    end if

    set exp = exp - 1

    set number = number + 2^exp

    end while

    If you follow this for .3 you get:

    .01001100110011001100110011... ad nauseum

    Compare this to the process of converting .1 from base 3 into base 10 (.1 in base 3 is the ratio 1/3)

    A common question when this explanation is given is: 3 can be represented in base 2, and -1 can be represented, so why doesn't the floating point data type represent the value as "3 * 10^-1" The answer as I understand it is that floats are highly optimized for storage space, so for the same number of bits, you would get less range of representable numbers from a float specified this way. Feel free to correct me on this if there were other considerations I'm forgetting.



    Dan Guzman - Not the MVP (7/22/2010)
    All questions have to be prefaced by Server version and 'according to MS Docs' or 'my own personal opinion based on how much detail I felt like digging into at the time.'