

Mr or Mrs. 500
Group: General Forum Members
Last Login: Thursday, March 27, 2014 8:16 AM
Points: 541,
Visits: 282


Dan Guzman  Not the MVP (12/30/2009) Great explanation Scott, that's very detailed and instructive. Unfortunately it does not address the behavior found by KevinC. Why is not the behavior you describe consistent in a given range of numbers?
Dan
It haven't checked, but I am pretty sure it does address the behaviour found  because the binary (IEEE) representation of these numbers will not be "consistent".
Basically it boils dow to "Don't use ROUND() with doubleprecision"
Kelsey Thornton MBCS CITP




SSC Veteran
Group: General Forum Members
Last Login: Tuesday, March 29, 2016 2:01 AM
Points: 295,
Visits: 214


Good to know.
oracle behaves the same.
declare round1a binary_float := 165.755; round2a binary_float := 165.555; round1b float := 165.755; round2b float := 165.555; begin dbms_output.PUT_LINE (ROUND(round1a,2)); dbms_output.PUT_LINE (ROUND(round2a,2)); dbms_output.PUT_LINE (ROUND(round1b,2)); dbms_output.PUT_LINE (ROUND(round2b,2)); end;
165.76 165.55 165.76 165.56
Rob




SSCrazy
Group: General Forum Members
Last Login: Wednesday, August 24, 2016 7:06 AM
Points: 2,925,
Visits: 1,380


I'll try this again. Intel processors (and probably most other computer hardware) use IEEE standard floating point representations. Single precision (REAL) uses four bytes and provides 24 bits to represent the value. Double precision (FLOAT) uses eight bytes and provides 53 bits for the value. The rounding issue discussed in this post is not due to the ROUND() function, it happens when the character string "165.755" is first converted to floating point. This means it should work the same in any programming environment on any computer hardware using the same IEEE standards.
Given the character string "165.755", the system computes the floating point value. Internally in the CPU, I believe calculations are done to 80 bit accuracy. The result is then shortened (with rounding) to 24 or 53 bits, depending on the resolution. In the case of 165.755, the 25th bit is a 1 so the singleprecision value is rounded up and is a little larger than 165.755. The 54th bit is a 0, so the doubleprecision value is rounded down and is a little smaller than 165.755. This happens when the characters "165.755" are first converted to a floating point representation and have nothing to do with the ROUND() function.
This is why floating point types are called approximate types, and are not used where the rounding in the last decimal place is critical. If you need exact decimal values you need to use DECIMAL, NUMERIC, or MONEY. The behavior may look random, but it is completely consistent and predictable. You just need to get out paper and pencil and figure out the binary representation of all the floating point values you plan to use to 80 bits or so.




Ten Centuries
Group: General Forum Members
Last Login: Monday, August 31, 2015 7:57 AM
Points: 1,156,
Visits: 598


Nice one Scott.
What you don't know won't hurt you but what you know will make you plan to know better




SSCrazy
Group: General Forum Members
Last Login: Thursday, August 25, 2016 7:13 AM
Points: 2,674,
Visits: 597


Thank you for a great qotd and explanation! Lesson learned.
/Håkan Winther MCITP:Database Developer 2008 MCTS: SQL Server 2008, Implementation and Maintenance MCSE: Data Platform




SSCommitted
Group: General Forum Members
Last Login: Thursday, August 18, 2016 2:14 PM
Points: 1,604,
Visits: 447


Scott, very fine explantion.
 Mark D Powell 




SSCommitted
Group: General Forum Members
Last Login: 2 days ago @ 6:26 PM
Points: 1,794,
Visits: 2,531


I'm honestly surprised at the number of people here who were not aware of the approximate nature of IEEE floatingpoint numbers. That information was in every programming class I took and every programming book I read. (It's even in BOL: http://msdn.microsoft.com/enus/library/ms187912.aspx  read the "Using float and real Data" section).
As a result, the first choice I was immediately able to eliminate was the "correct" one. I have to admit, though: I didn't want to go throught the same level of work as Scott did (great explanation, by the way); nor did I want to "cheat" by copypasteexecute, so I guessed (correctly, as it turned out).
This is not a criticism of anyone's knowledge  it is instead a vote of applause for this question. It brings out a limitation of a commonlyused data type, of which many were not aware. This is an example of a question that both tests and educates.
Plus, it reinforces my natural instinct to avoid float whenever possible...




Hall of Fame
Group: General Forum Members
Last Login: Saturday, June 6, 2015 9:44 PM
Points: 3,450,
Visits: 4,408


sknox (1/11/2010) I have to admit, though: I didn't want to go throught the same level of work as Scott did (great explanation, by the way); nor did I want to "cheat" by copypasteexecute, so I guessed (correctly, as it turned out). I cheated because I knew about the incomplete correspondence between decimals and binaries, and did not want to guess. But now I have found a great website for decimaltoIEEE754 conversion: http://www.binaryconvert.com
This site shows hexadecimal and binary representation of an entered number, as well as the most accurate result of the conversion from this binary back to decimal.
Here is the result for 165.755 (double precision): 0x4064B828F5C28F5C 01000000 01100100 10111000 00101000 11110101 11000010 10001111 01011100 The result of backward conversion to decimal: 1.65754999999999995452526491135E2
The same for 165.555: 0x4064B1C28F5C28F6 01000000 01100100 10110001 11000010 10001111 01011100 00101000 11110110 1.65555000000000006821210263297E2
With these values the correct answer becomes obvious :)




Mr or Mrs. 500
Group: General Forum Members
Last Login: Thursday, March 27, 2014 8:16 AM
Points: 541,
Visits: 282


Now I have found a great website for decimaltoIEEE754 conversion: http://www.binaryconvert.comThis site shows hexadecimal and binary representation of an entered number, as well as the most accurate result of the conversion from this binary back to decimal.
Great resource  Thanks!
Kelsey Thornton MBCS CITP



