MATLAB Central Newsreader - decimal precision

decimal precision

sujata — Wed, 04 Nov 2009 12:56:02 -0500

Hi

I have a question. I have the following calculation

(1+number1)/(1+number2) - 1

The number 1 and number 2 are scalars with 16 decimals

when I calculate the upper formula in both excel and matlab with the same number1 and number2, I get a difference on the last three decimals.

Why is this so? And what should I do that they are exact the same.

Re: decimal precision

dpb — Wed, 04 Nov 2009 15:01:36 -0500

sujata wrote:
> Hi
>
> I have a question. I have the following calculation
>
> (1+number1)/(1+number2) - 1
>
>
> The number 1 and number 2 are scalars with 16 decimals
>
> when I calculate the upper formula in both excel and matlab with the
> same number1 and number2, I get a difference on the last three decimals.
>
> Why is this so? And what should I do that they are exact the same.

Artifact of floating point representation.

If indeed n1 == n2 precisely, the result should be 1; otoh if n1 and n2
are obtained in different manner (such as one is typed or read in while
the other is computed, for example) they may differ in internal
representation in a last bit or two and so the result isn't identical.

In double precision the number of mantissa bit works out to 15/16
decimal digits of precision and since internal representation is in
binary fractions instead of decimal this is an approximate number and
will differ depending on rounding to nearest representable value. Hence
it's probable that the "1's" in the numerator and denominator underflow
but again depending on whether n1==n2 identically, it's possible that
for some values of n one does and one doesn't again owing to rounding.

The general answer to the last plaintive plea is "write code that is
robust to floating point approximations" -- the answer as to how to do
that in general is the subject of many tomes and is "depends"...

See the ML wiki FAQ for discussion and link to a paper by Goldberg
that's essentially required reading for computing...

--

Re: decimal precision

someone — Wed, 04 Nov 2009 15:21:01 -0500

dpb <none@non.net> wrote in message <hcs5a8$gn8$1@news.eternal-september.org>...
> sujata wrote:
> > Hi
> >
> > I have a question. I have the following calculation
> >
> > (1+number1)/(1+number2) - 1
> >
> >
> > The number 1 and number 2 are scalars with 16 decimals
> >
> > when I calculate the upper formula in both excel and matlab with the
> > same number1 and number2, I get a difference on the last three decimals.
> >
> > Why is this so? And what should I do that they are exact the same.
>
> Artifact of floating point representation.
>
> If indeed n1 == n2 precisely, the result should be 1; otoh if n1 and n2
> are obtained in different manner (such as one is typed or read in while
> the other is computed, for example) they may differ in internal
> representation in a last bit or two and so the result isn't identical.
>
> In double precision the number of mantissa bit works out to 15/16
> decimal digits of precision and since internal representation is in
> binary fractions instead of decimal this is an approximate number and
> will differ depending on rounding to nearest representable value. Hence
> it's probable that the "1's" in the numerator and denominator underflow
> but again depending on whether n1==n2 identically, it's possible that
> for some values of n one does and one doesn't again owing to rounding.
>
> The general answer to the last plaintive plea is "write code that is
> robust to floating point approximations" -- the answer as to how to do
> that in general is the subject of many tomes and is "depends"...
>
> See the ML wiki FAQ for discussion and link to a paper by Goldberg
> that's essentially required reading for computing...

Those sources are in the answer to Q6.1 of the MATLAB FAQ at:

http://matlabwiki.mathworks.com/MATLAB_FAQ