• Sergiy (8/31/2016)


    I'll have to disagree with you there, ol' friend. Automatic truncation to a scale of 6 isn't correct. At least not in my book. If a dollar store calculator did such a thing, I'd take the time to get a refund.

    You disagree because you believe in that truncation.

    The fact is - there is no any truncation.

    Jeff, run this script:

    DECLARE @D DECIMAL (38, 35), @F FLOAT

    SET @D = 1.001 SET @F = 1.001

    SELECT 0 [Number of Operations],

    @D,

    SQL_VARIANT_PROPERTY(@D, 'Precision') DecimalPrecision,

    SQL_VARIANT_PROPERTY(@D, 'Scale') DecimalScale,

    @F,

    SQL_VARIANT_PROPERTY(@F, 'Precision') FloatPrecision,

    SQL_VARIANT_PROPERTY(@F, 'Scale') FloatScale

    UNION

    SELECT 1 ,

    @D*@D,

    SQL_VARIANT_PROPERTY(@D*@D, 'Precision') DecimalPrecision,

    SQL_VARIANT_PROPERTY(@D*@D, 'Scale') DecimalScale,

    @F*@F,

    SQL_VARIANT_PROPERTY(@F*@F, 'Precision') FloatPrecision,

    SQL_VARIANT_PROPERTY(@F*@F, 'Scale') FloatScale

    UNION

    SELECT 2 ,

    @D*@D*@D,

    SQL_VARIANT_PROPERTY(@D*@D*@D, 'Precision') DecimalPrecision,

    SQL_VARIANT_PROPERTY(@D*@D*@D, 'Scale') DecimalScale,

    @F*@F*@F,

    SQL_VARIANT_PROPERTY(@F*@F*@F, 'Precision') FloatPrecision,

    SQL_VARIANT_PROPERTY(@F*@F*@F, 'Scale') FloatScale

    UNION

    SELECT 3 ,

    @D*@D*@D*@D,

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D, 'Precision') DecimalPrecision,

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D, 'Scale') DecimalScale,

    @F*@F*@F*@F,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F, 'Precision') FloatPrecision,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F, 'Scale') FloatScale

    UNION

    SELECT 4 ,

    CONVERT(DECIMAL (38, 35), @D*@D*@D*@D*@D),

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D*@D, 'Precision') DecimalPrecision,

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D*@D, 'Scale') DecimalScale,

    @F*@F*@F*@F*@F,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F*@F, 'Precision') FloatPrecision,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F*@F, 'Scale') FloatScale

    UNION

    SELECT 5 ,

    CONVERT(DECIMAL (38, 35), @D*@D*@D*@D*@D*@D),

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D*@D*@D, 'Precision') DecimalPrecision,

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D*@D*@D, 'Scale') DecimalScale,

    @F*@F*@F*@F*@F*@F,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F*@F*@F, 'Precision') FloatPrecision,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F*@F*@F, 'Scale') FloatScale

    UNION

    SELECT 6 ,

    CONVERT(DECIMAL (38, 35), @D*@D*@D*@D*@D*@D*@D),

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D*@D*@D*@D, 'Precision') DecimalPrecision,

    SQL_VARIANT_PROPERTY(@D*@D*@D*@D*@D*@D*@D, 'Scale') DecimalScale,

    @F*@F*@F*@F*@F*@F*@F,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F*@F*@F*@F, 'Precision') FloatPrecision,

    SQL_VARIANT_PROPERTY(@F*@F*@F*@F*@F*@F*@F, 'Scale') FloatScale

    See?

    There is no any truncation to scale of 6.

    Each of additional operation eats out another 4 digits from the scale.

    Why 4?

    Because the max number DECIMAL(38,35) can accommodate is 999.9(9)

    Multiplying these 2 numbers gives 999999.9(9) - which after inevitable rounding (because the result exceeds 38 digits limitation) gives 1000000 - 7 digits.

    So, to prevent arithmetic overflow of 2 decimal number multiplication we need to reserve for the result 7 digits:

    (p1 - s1 ) + (p2 - s2) + 1 = (38-35) + (38-35) + 1 = 7

    But we cannot increase the total precision of the number, because we are at its max.

    So we have to reduce the number of significant digits after the decimal point:

    38 - 7 = 31

    That's the decimal scale you see after 1 operation in the script above.

    2nd multiplication will remove other 4 digits, and so on, until only 6 "scale" digits left.

    At this point, MS decided that they cannot go any lower in terms of decimal scale and allowed arithmetic overflow to happen rather than eliminate the decimal scale completely.

    P.S. Granny's calculator is not any better than SQL Server in terms of decimal multiplication/division.

    It only looks better because you don't see the whole picture.

    Good calculators always have spare precision depth which is not shown on the display.

    10 digit calculator usually holds 16 or 18 digits in memory, but displays only 10 of them.

    If you run a test on a calculator having this in mind you pretty quickly will find out that calculators (except those scientific ones which do floating point computations) do decimal math with the same systematic errors as

    SQL Server.

    _____________
    Code for TallyGenerator