<a href="http://g.adspeed.net/ad.php?do=clk&zid=15220&wd=468&ht=60&pair=as" target="_blank"><img style="border:0px;" src="http://g.adspeed.net/ad.php?do=img&zid=15220&wd=468&ht=60&pair=as" alt="i" width="468" height="60"/></a>
Log in
::
Register
::
Not logged in
Home
Tags
Articles
Editorials
Stairways
Forums
Scripts
Videos
Blogs
QotD
Books
Ask SSC
SQL Jobs
Training
Authors
About us
Contact us
Newsletters
Write for us
<a href="http://g.adspeed.net/ad.php?do=clk&zid=15491&wd=120&ht=300&pair=as" target="_top"><img style="border:0px;" src="http://g.adspeed.net/ad.php?do=img&zid=15491&wd=120&ht=300&pair=as" alt="i" width="120" height="300"/></a>
Recent Posts
Recent Posts
Popular Topics
Popular Topics
Home
Search
Members
Calendar
Who's On
Home
»
Article Discussions
»
Article Discussions by Author
»
Discuss content posted by brewmanz
»
SUM of FLOAT inconsistency
15 posts, Page 2 of 2
««
1
2
SUM of FLOAT inconsistency
Rate Topic
Display Mode
Topic Options
Author
Message
Anipaul
Anipaul
Posted Monday, October 13, 2008 11:12 PM
SSCarpal Tunnel
Group: General Forum Members
Last Login: 2 days ago @ 2:50 AM
Points: 4,785,
Visits: 1,334
Nice question......
Post #585202
Carlo Romagnano
Carlo Romagnano
Posted Tuesday, October 14, 2008 12:38 AM
SSCommitted
Group: General Forum Members
Last Login: Yesterday @ 2:04 AM
Points: 1,968,
Visits: 1,819
brewmanz (10/13/2008)
T
It always is correct WITH THIS SET OF DATA. Try adding a couple of zero(e)s at the end of each of the 3 numbers and watch "Msg 8115, Level 16, State 6, Line 12 - Arithmetic overflow error converting float to data type numeric." appear.
The FLOAT still works, and produces even stranger (but predictable) results of 1000164 and 1000192.
I prefer an overflow error instead of wrong data. You can specify a precision of 38 digits
From BOL:
Numeric data types that have fixed precision and scale.
decimal[ (p[ , s] )] and numeric[ (p[ , s] )]
Fixed precision and scale numbers. When maximum precision is used, valid values are from - 10^38 +1 through 10^38 - 1. The SQL-92 synonyms for decimal are dec and dec(p, s). numeric is functionally equivalent to decimal.
p (precision)
The maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision must be a value from 1 through the maximum precision of 38. The default precision is 18.
s (scale)
The maximum number of decimal digits that can be stored to the right of the decimal point. Scale must be a value from 0 through p. Scale can be specified only if precision is specified. The default scale is 0; therefore, 0 <= s <= p. Maximum storage sizes vary, based on the precision.
Post #585224
Chris.Strolia-Davis
Chris.Strolia-Davis
Posted Tuesday, October 14, 2008 6:44 AM
SSC Rookie
Group: General Forum Members
Last Login: Tuesday, May 24, 2011 7:50 AM
Points: 41,
Visits: 83
brewmanz (10/13/2008)
Please don't tell my daughter. I take great delight in proofreading her work and finding mistakes (she works for a brochure publishing company). Sadly, it seems that I am capable of making mistakes, too. Soon I'll be into double figures this century
Ha ha, it happens to the best of us my friend. Seems like life enjoys dishing out the humble pie every now and then.
Thanks again for the article. It's good for developers to be aware of, and judging by the results of the quiz, there are quite a few out there who could use the enlightenment.
Post #585404
Todd Carrier
Todd Carrier
Posted Wednesday, October 15, 2008 3:31 PM
Ten Centuries
Group: General Forum Members
Last Login: Tuesday, May 14, 2013 2:54 PM
Points: 1,049,
Visits: 633
Brewmanz... keep them coming. That was great, and exactly what we all need! Good explanation, too. I found that it was dependent upon order, but could not figure out why. A+++ QotD!
Todd Carrier
MCITP - Database Administrator
Post #586606
Nakul Vachhrajani
Nakul Vachhrajani
Posted Monday, August 09, 2010 8:06 AM
Ten Centuries
Group: General Forum Members
Last Login: 2 days ago @ 9:52 AM
Points: 1,277,
Visits: 1,608
You can validate this using Excel as well. You will find the same thing in Excel also (Refer attachment)
Excellent post!
Thanks & Regards,
Nakul Vachhrajani.
http://beyondrelational.com/modules/2/blogs/77/nakuls-blog.aspx
Be courteous. Drive responsibly.
Follow me on
Twitter:
@nakulv_sql
Google Plus:
+Nakul
Post Attachments
SUM of FLOAT inconsistency.xlsx
(
4 views,
8.99 KB
)
Post #965975
« Prev Topic
|
Next Topic »
15 posts, Page 2 of 2
««
1
2
Permissions
You
cannot
post new topics.
You
cannot
post topic replies.
You
cannot
post new polls.
You
cannot
post replies to polls.
You
cannot
edit your own topics.
You
cannot
delete your own topics.
You
cannot
edit other topics.
You
cannot
delete other topics.
You
cannot
edit your own posts.
You
cannot
edit other posts.
You
cannot
delete your own posts.
You
cannot
delete other posts.
You
cannot
post events.
You
cannot
edit your own events.
You
cannot
edit other events.
You
cannot
delete your own events.
You
cannot
delete other events.
You
cannot
send private messages.
You
cannot
send emails.
You
may
read topics.
You
cannot
rate topics.
You
cannot
vote within polls.
You
cannot
upload attachments.
You
may
download attachments.
You
cannot
post HTML code.
You
cannot
edit HTML code.
You
cannot
post IFCode.
You
cannot
post JavaScript.
You
cannot
post EmotIcons.
You
cannot
post or upload images.
Copyright © 2002-2013 Simple Talk Publishing. All Rights Reserved.
Privacy Policy.
Terms of Use.
Report Abuse.
