SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Five percent rounded down


Five percent rounded down

Author
Message
mhtanner
mhtanner
Valued Member
Valued Member (62 reputation)Valued Member (62 reputation)Valued Member (62 reputation)Valued Member (62 reputation)Valued Member (62 reputation)Valued Member (62 reputation)Valued Member (62 reputation)Valued Member (62 reputation)

Group: General Forum Members
Points: 62 Visits: 129
Comments posted to this topic are about the item Five percent rounded down
kapil_kk
kapil_kk
Hall of Fame
Hall of Fame (3.6K reputation)Hall of Fame (3.6K reputation)Hall of Fame (3.6K reputation)Hall of Fame (3.6K reputation)Hall of Fame (3.6K reputation)Hall of Fame (3.6K reputation)Hall of Fame (3.6K reputation)Hall of Fame (3.6K reputation)

Group: General Forum Members
Points: 3600 Visits: 2766
god start with a coffee +2 :-)

_______________________________________________________________
To get quick answer follow this link:
http://www.sqlservercentral.com/articles/Best+Practices/61537/
Raghavendra Mudugal
Raghavendra Mudugal
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2520 Visits: 2958
Very interesting, thank you for the question.

(and Happy Dasara to all)

ww; Raghu
--
The first and the hardest SQL statement I have wrote- "select * from customers" - and I was happy and felt smart.
Tom Thomson
Tom Thomson
SSCoach
SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)

Group: General Forum Members
Points: 17528 Visits: 12331
Nice question.

The trick of adding 0.000000001 to the multiplicand might work sometimes, but it isn't the best way to handle this.

Until someone introduces proper modern floating point into SQL (which would eliminate the problem completely) the way to handle this is to use round, which means going via numeric. So in this case, change the calculation for Quota3 to be either
select @Quota3 = round(@Value*cast(1.05 as numeric(5,2)) ,0);
or
select @Quota3 = round(cast((cast(@Value as float) * 1.05) as numeric(38,27)),0);
and the problem goes away. (and of course there should be similar changes in the calculations of the other two numbers).
Note that it is necessary, in the second version, to change from using real to using float if you want it to work whatever your integer is. Real (float(24)) is not a sensible type to use for this, since it can't represent all 32 bit integers; so it makes sense to change real to float, which can, as well as using round.

But proper modern floating point would be much much better. It's about time we were hearing something about it for SQL Standards, as it's now 50 months since IEEE 754-2008 was published.

Tom

Michael Riemer
Michael Riemer
SSCrazy
SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)

Group: General Forum Members
Points: 2932 Visits: 643
Thanks for the question, it is a great point to learn from! I hadn't seen this before, but know there are always issues with real and float numbers so had to check, and the title kind of gave me a clue that something wouldn't be right.

I guess you only really know this if it has bitten you already!
Koen Verbeeck
Koen Verbeeck
SSC-Dedicated
SSC-Dedicated (34K reputation)SSC-Dedicated (34K reputation)SSC-Dedicated (34K reputation)SSC-Dedicated (34K reputation)SSC-Dedicated (34K reputation)SSC-Dedicated (34K reputation)SSC-Dedicated (34K reputation)SSC-Dedicated (34K reputation)

Group: General Forum Members
Points: 34463 Visits: 13270
Interesting question, thanks.
I do wonder how people are supposed to get this right without running the script.


How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?
My blog at SQLKover.

MCSE Business Intelligence - Microsoft Data Platform MVP
palotaiarpad
palotaiarpad
SSCrazy
SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)

Group: General Forum Members
Points: 2215 Visits: 783
Lets play a bit:
Select cast(100 as real) * 1.05,cast(cast(100 as real) * 1.05 as int);
-- Result is 105, 104



The result is very strange for me. Does SQL Server the calculation from inside to outside or not? w00t
Raghavendra Mudugal
Raghavendra Mudugal
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2520 Visits: 2958
palotaiarpad (10/24/2012)
Lets play a bit:
Select cast(100 as real) * 1.05,cast(cast(100 as real) * 1.05 as int);
-- Result is 105, 104



The result is very strange for me. Does SQL Server the calculation from inside to outside or not? w00t


...change the second "real" to "numeric" it gives same as 105...

(as it says in BOL "Approximate-number data types for use with floating point numeric data. Floating point data is approximate; therefore, not all values in the data type range can be represented exactly. ")

ww; Raghu
--
The first and the hardest SQL statement I have wrote- "select * from customers" - and I was happy and felt smart.
malleswarareddy_m
malleswarareddy_m
SSCrazy
SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)

Group: General Forum Members
Points: 2347 Visits: 1189
Nice question.

Malleswarareddy
I.T.Analyst
MCITP(70-451)
palotaiarpad
palotaiarpad
SSCrazy
SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)SSCrazy (2.2K reputation)

Group: General Forum Members
Points: 2215 Visits: 783
My problem is, that the first expression evaluates to 105, and if i cast the 105 to int it should remain 105 and not less.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search