Nice, even if a rather simple question (but it does take one back to the basics and because of that a GOOD QOD) ... and yes I selected the correct answer.

If everything seems to be going well, you have obviously overlooked something.

Group: General Forum Members Last Login: Yesterday @ 10:42 AM Points: 3,868,Visits: 5,024

Good "back to Basics" question, Paul Thanks

____________________________________________ Space, the final frontier? not any more... All limits henceforth are self-imposed. “libera tute vulgaris ex”

Group: General Forum Members Last Login: Yesterday @ 8:59 AM Points: 5,930,Visits: 8,181

Happy new year everyone! ;)

It doesn't happen very often, but today I find myself in disagreement with Paul. The correct answer is (or should be) DECIMAL.

The "precision" mentioned on the BOL page about floating point data is not the number of decimal positions represented internally, but the number of decimal positions that can be trusted to be correct. The internal representation is different (base-2 to be exact). As mentioned on the top of the BOL page: "not all values in the data type range can be represented exactly". The value 0.1234567 is one of the many values that can not be represented exactly. When trying the code snippet in SSMS, this is carefully hidden - because SSMS knows about the number of "trustworthy" positions for the REAL data type and shows only that number of positions. Other front-end tools might not be so smart. And front-end or back-end applications that do further calculations on the number do get the internal representation, which in the case of the REAL data is not exactly correct.

Here is a modification of Paul's code that tricks SSMS into revealing a more accurate representation of the values internally stored; it shows that the DECIMAL value is indeed exact, but the REAL value is not.

SELECT [decimal] = CAST(@n1 * @n2 AS decimal(38,36)), [real] = CAST(@n3 * @n4 AS decimal(38,36))

Group: General Forum Members Last Login: Yesterday @ 1:32 PM Points: 11,194,Visits: 11,112

Hugo Kornelis (1/3/2012)It doesn't happen very often, but today I find myself in disagreement with Paul. The correct answer is (or should be) DECIMAL...

Some good points, and it is true (for example) that Query Analyzer against a SQL Server 2000 instance gives the result using REAL as 0.12345671 but no matter how you CAST the result, the result of the DECIMAL calculation is 0.123457 (i.e. rounded to six decimal places) so the REAL result is, in the wording of the question ("In the example below, which data type will give the most accurate answer?") more accurate. I don't see how the answer could possibly be DECIMAL (0.123457 decimal is much further from 0.1234567 (correct) than 0.12345670908689499 (real, unadjusted for precision))?

Using REAL in this case does give exactly the right displayed answer in SSMS (though not in other tools as you say) and when working with approximate data types we do need to be careful about the significant digits (as demonstrated by SSMS). Anyway, the main point of the question is for people to be aware of the limitations of the fixed-precision types, and in particular to avoid using excessive scale.

Group: General Forum Members Last Login: Yesterday @ 8:59 AM Points: 5,930,Visits: 8,181

OOPS!! Sorry, Paul - you are completely right.

I had missed the rounding of the decimal result; I thought that the decimal result was the exact and correct result. Even when running the code I posted above (that shows the extra decimals), I simply overlooked that the DECIMAL result was 0.123457 instead of 0.1234567. My bad; my apologies.

You cannot post new topics. You cannot post topic replies. You cannot post new polls. You cannot post replies to polls. You cannot edit your own topics. You cannot delete your own topics. You cannot edit other topics. You cannot delete other topics. You cannot edit your own posts. You cannot edit other posts. You cannot delete your own posts. You cannot delete other posts. You cannot post events. You cannot edit your own events. You cannot edit other events.

You cannot delete your own events. You cannot delete other events. You cannot send private messages. You cannot send emails. You may read topics. You cannot rate topics. You cannot vote within polls. You cannot upload attachments. You may download attachments. You cannot post HTML code. You cannot edit HTML code. You cannot post IFCode. You cannot post JavaScript. You cannot post EmotIcons. You cannot post or upload images.