• select cast(1.67574 as decimal(38,10)) = 1.675400000 which has precision 6 and scale 10, right?

    and

    select cast(10000 as decimal(38,10)) = 10000.0000000000 which has precision 1 and scale 10, right?

    So,

    [font="Courier New"]p1 - s1 + s2 + max(6, s1 + p2 + 1) =

    6 - 10 + 10 + max(6, 10 + 1 + 1) =

    6 + 12 = 18.[/font]

    And the result scale is max(6, s1 + p2 + 1) = max(6, 10 + 1 + 1) = 12.

    So, where is the rounding happening?

    In Dude76's step by step we see that d1 / d2 is being rounded and is not returning the promised 18 characters of precision and 12 positions of scale. Why?


    Steve Eckhart