December 17, 2007 at 11:10 pm
Hi,
Please execute the below statements in Query Analyzer.
1st one is returning 1528896000.0000000
2nd one is returning 1528896000.0000002
can anyone let me know some details on this
select convert(numeric(22,7),1200000000.0000000*1.27408)
SELECT convert(numeric(22,7),(1200000000.0000000*convert(float,(1.27408))))
December 18, 2007 at 3:46 am
Floats are inaccurate data types. Numerics are precise.
From Books online
The float and real data types are known as approximate data types. The behavior of float and real follows the IEEE 754 specification on approximate numeric data types.
Approximate numeric data types do not store the exact values specified for many numbers; they store an extremely close approximation of the value. For many applications, the tiny difference between the specified value and the stored approximation is not noticeable. At times, though, the difference becomes noticeable. Because of the approximate nature of the float and real data types, do not use these data types when exact numeric behavior is required, such as in financial applications, in operations involving rounding, or in equality checks. Instead, use the integer, decimal, money, or smallmoney data types.
The numeric data type can store a maximum of 38 digits, all of which can be to the right of the decimal point. The numeric data type stores an exact representation of the number; there is no approximation of the stored value.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
December 18, 2007 at 4:08 am
Hi,
Thanks a lot!
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply