Thread Subject: Inexact output

If I am doing a calculation, and the result is

9.699999999999999

when it should be exactly 9.7, what can I do to correct it?

Thanks!

"Paul " <palex71@hotmail.com> wrote in message <itu0eh$9hb$1@newscl01ah.mathworks.com>...
> If I am doing a calculation, and the result is
>
> 9.699999999999999
>
> when it should be exactly 9.7, what can I do to correct it?
>
> Thanks!
- - - - - - - - - - -
  Welcome to the binary world! Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly. What you see there is about as close as it can get. To get some idea of the problem try representing 1/3 as an exact decimal fraction.

Roger Stafford

"Roger Stafford" wrote in message <itu0vd$asq$1@newscl01ah.mathworks.com>...
> "Paul " <palex71@hotmail.com> wrote in message <itu0eh$9hb$1@newscl01ah.mathworks.com>...
> > If I am doing a calculation, and the result is
> >
> > 9.699999999999999
> >
> > when it should be exactly 9.7, what can I do to correct it?
> >
> > Thanks!
> - - - - - - - - - - -
> Welcome to the binary world! Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly. What you see there is about as close as it can get. To get some idea of the problem try representing 1/3 as an exact decimal fraction.
>
> Roger Stafford


Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true). Any ideas?

Thanks again.

"Paul " <palex71@hotmail.com> wrote in message <itu0eh$9hb$1@newscl01ah.mathworks.com>...
> If I am doing a calculation, and the result is
>
> 9.699999999999999
>
> when it should be exactly 9.7, what can I do to correct it?
>
> Thanks!
Hi paul,
  You should round it off. Otherwise put the condition statement like this
if abs(a-b)<tol instead of if a==b,
where tol is tolerance value, say, 10^-5 etc.
Or try this-
a1=sprintf('%0.1d',9.6999999999999);
a2=sprintf('%0.1d',9.7)
if strcmp(a1,a2).....

HTH,
Krishna

"Paul " <palex71@hotmail.com> wrote in message <itufrk$gc3$1@newscl01ah.mathworks.com>...
> Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true). Any ideas?
- - - - - - - -
  There needs to be a careful distinction made between two concepts concerning a numerical result in matlab. There is the way the number is displayed to the user and there is the binary number actually stored in the computer's memory, and these are not necessarily the same. (What you see is not always exactly what you get.) It is only the stored number that is used for computations, with the displayed value being provided purely for the benefit of the user.

  In the case of a number entered as 9.7, the computer will actually store this binary number:

 1001.1011001100110011001100110011001100110011001100110

It is the binary double precision's closest approximation to 9.7 and as you can see from the repeating pattern, no matter how many bits we were to provide the computer it would always be in error by some non-zero amount with the pattern continuing out indefinitely far.

  As for how such a number is displayed to the user, that all depends on how the display format is set. With 'format short' it appears as:

 9.7000
 
With (my) 'format long' it is:

 9.70000000000000

With fprintf('%18.16f\n',9.7) it is:

 9.6999999999999993

With 'format hex' it is:

 4023666666666666

This last, (in IEEE 754 hexadecimal format,) is the only one that gives the true value of the binary quantity that is stored.

  If as you stated earlier the number is the result of some computation, it is quite likely that it will have suffered a few bits worth of round off error at the low end, in which case the last three formats might be slightly different than above. In the case of the 'format short' if the correct answer is 9.7 this format would continue to give 9.7000 unless some very large errors had occurred in the computation. On the other hand, if the exact answer is supposed to be, say, 9.7000125, then the format short would continue to give you what would then be an erroneous 9.7000 .

  I hope that somewhere in the above meandering discussion there lies an answer to your question about how to "convert" the output to match another value - or rather, an explanation for why it would be almost impossible for such a conversion attempt to always succeed. I think you will have to give up the notion of always getting exact matches with numbers that have undergone different procedures and settle for only approximate matches. This is inherent in the very nature of numerical computations performed by digital computers.

Roger Stafford

On Jun 23, 6:36 am, "Paul " <pale...@hotmail.com> wrote:
> "Roger Stafford" wrote in message <itu0vd$as...@newscl01ah.mathworks.com>...
> > "Paul " <pale...@hotmail.com> wrote in message <itu0eh$9h...@newscl01ah.mathworks.com>...
> > > If I am doing a calculation, and the result is
>
> > > 9.699999999999999
>
> > > when it should be exactly 9.7, what can I do to correct it?
>
> > > Thanks!
> > - - - - - - - - - - -
> >   Welcome to the binary world!  Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly.  What you see there is about as close as it can get.  To get some idea of the problem try representing 1/3 as an exact decimal fraction.
>
> > Roger Stafford
>
> Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true).  Any ideas?

Sure. Learn how to use insexact aritmetics:

http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems

Rune

Rune Allnor <allnor@tele.ntnu.no> wrote in message <0a83bac8-76a9-4786-810f-64eb0eee0eed@j23g2000yqc.googlegroups.com>...
> On Jun 23, 6:36 am, "Paul " <pale...@hotmail.com> wrote:
> > "Roger Stafford" wrote in message <itu0vd$as...@newscl01ah.mathworks.com>...
> > > "Paul " <pale...@hotmail.com> wrote in message <itu0eh$9h...@newscl01ah.mathworks.com>...
> > > > If I am doing a calculation, and the result is
> >
> > > > 9.699999999999999
> >
> > > > when it should be exactly 9.7, what can I do to correct it?
> >
> > > > Thanks!
> > > - - - - - - - - - - -
> > >   Welcome to the binary world!  Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly.  What you see there is about as close as it can get.  To get some idea of the problem try representing 1/3 as an exact decimal fraction.
> >
> > > Roger Stafford
> >
> > Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true).  Any ideas?
>
> Sure. Learn how to use insexact aritmetics:
>
> http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems
>
> Rune

Thanks everyone for the helpful replies. I did use if abs(x-y)<tol.... and it appears to be working.

Regards.

Besides binary finite representation, keep in mind that 9.699... with and infinite number of 9's is exactly equal to 9.7


"Paul " <palex71@hotmail.com> wrote in message <iu0dg4$bo4$1@newscl01ah.mathworks.com>...
> Rune Allnor <allnor@tele.ntnu.no> wrote in message <0a83bac8-76a9-4786-810f-64eb0eee0eed@j23g2000yqc.googlegroups.com>...
> > On Jun 23, 6:36 am, "Paul " <pale...@hotmail.com> wrote:
> > > "Roger Stafford" wrote in message <itu0vd$as...@newscl01ah.mathworks.com>...
> > > > "Paul " <pale...@hotmail.com> wrote in message <itu0eh$9h...@newscl01ah.mathworks.com>...
> > > > > If I am doing a calculation, and the result is
> > >
> > > > > 9.699999999999999
> > >
> > > > > when it should be exactly 9.7, what can I do to correct it?
> > >
> > > > > Thanks!
> > > > - - - - - - - - - - -
> > > >   Welcome to the binary world!  Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly.  What you see there is about as close as it can get.  To get some idea of the problem try representing 1/3 as an exact decimal fraction.
> > >
> > > > Roger Stafford
> > >
> > > Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true).  Any ideas?
> >
> > Sure. Learn how to use insexact aritmetics:
> >
> > http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems
> >
> > Rune
>
> Thanks everyone for the helpful replies. I did use if abs(x-y)<tol.... and it appears to be working.
>
> Regards.