By Roy Carlson, 2005/04/04

Rcently, I was asked to pull some statistics from a SQL database. Only minimal information was available on the data itself, and the task required determining percentages and creating trends from the data. It was relatively easy to pull out the data and perform calculations, but the results looked funny when I had finished. So I dug further in.

Some of the numbers were integers from counts of row occurrences. A simple Example would be when I had to find the answer to this: "How many times did customer A order during a year, and what percent of the companies total orders comprised customer A.?" Giving some numbers to this example, if Customer A had 16 orders out of 140 total orders for all customers, then to find the percentage of Customer A's orders it would be (16/140)*100=11.43%. If we promised to give all salespersons who had customers ordering at least 10% of our total business a special bonus, this salesperson would get zip with my initial calculations. That's because in SQL Server, select (16/140)*100”=0.

However since our actual equation was much more complex, we did not see that SQL returned 0 after truncating some of our calculations. The calculations using integers, decimal, etc. do not always return the expected result. This is especially true with complex calculations or functions where the error can remain unseen.

Try the following example in query analyzer:

```select 6/9
select (6 + .0)/9
select (6 - .0)/9
select (6 * 1.0)/9
```

The first equation result is truncated to the nearest integer, which is 0. The last three equations return the correct amount. You can see that if this calculation is part of a large complex equation you can produce major errors. And you might not easily find them.

The second example even has a float declaration and a decimal but gives two different results. Try this second example in query analyzer:

```DECLARE @angle1 float
SET @angle1 = (.222456 + 6/9)
SELECT 'The SINE of the angle1 is: ' + CONVERT(varchar,SIN(@angle1))
GO
DECLARE @angle2 float
SET @angle2 = (.222456 + (6 + .0)/9)
SELECT 'The SINE of the angle2 is: ' + CONVERT(varchar,SIN(@angle2))
GO
```

When the equations are complex it is very easy to miss this conversion error. There are notations in SQL arithmetic stating that conversion to decimal may not occur.

Great! We could convert all of our table data to decimals and all numbers to decimal.

### The Easier Way

Have a good idea what of type is your data: integer, decimal, float....whatever. And most importantly, make sure your data is returning the results you expect. It was very easy to add or subtract (.0) or multiply by (1.0) to the equation at the point of error to force a decimal, which is easier than changing all the data.

6/9=0 is ok only when you are truncating to the nearest integer. But never assume the result of an equation is correct without testing it for truncation or rounding results that could cause errors.

Roy Carlson

Illinois Student Assistance Commission

Total article views: 6637 | Views in the last 30 days: 1

Related Articles
FORUM

### Select Decimal

Need Decimal() function

FORUM

### Displaying Fewer Decimal Places

Displaying fewer decimal places

FORUM

### Rounding Decimal value get wrong result

Rounding Decimal value get wrong result

FORUM

### Remove Decimals Without Rounding

How do I remove decimals

FORUM

### I do i express this equation in a query

express equation in query column, pythagoras, distance calculation

Tags

## Join the most active online SQL Server Community

### SQL knowledge, delivered daily, free:

#### You make SSC a better place

As a member of SQLServerCentral, you get free access to loads of fresh content: thousands of articles and SQL scripts, a library of free eBooks, a weekly database news roundup, a great Q & A platformâ€¦ And itâ€™s our huge, buzzing community of SQL Server Professionals that makes it such a success.