http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628
MATLAB Central Newsreader  Inexact output
Feed for thread: Inexact output
enus
©19942015 by MathWorks, Inc.
webmaster@mathworks.com
MATLAB Central Newsreader
http://blogs.law.harvard.edu/tech/rss
60
MathWorks
http://www.mathworks.com/images/membrane_icon.gif

Thu, 23 Jun 2011 00:13:05 +0000
Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842505
Paul
If I am doing a calculation, and the result is<br>
<br>
9.699999999999999<br>
<br>
when it should be exactly 9.7, what can I do to correct it?<br>
<br>
Thanks!

Thu, 23 Jun 2011 00:22:05 +0000
Re: Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842506
Roger Stafford
"Paul " <palex71@hotmail.com> wrote in message <itu0eh$9hb$1@newscl01ah.mathworks.com>...<br>
> If I am doing a calculation, and the result is<br>
> <br>
> 9.699999999999999<br>
> <br>
> when it should be exactly 9.7, what can I do to correct it?<br>
> <br>
> Thanks!<br>
          <br>
Welcome to the binary world! Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly. What you see there is about as close as it can get. To get some idea of the problem try representing 1/3 as an exact decimal fraction.<br>
<br>
Roger Stafford

Thu, 23 Jun 2011 04:36:05 +0000
Re: Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842522
Paul
"Roger Stafford" wrote in message <itu0vd$asq$1@newscl01ah.mathworks.com>...<br>
> "Paul " <palex71@hotmail.com> wrote in message <itu0eh$9hb$1@newscl01ah.mathworks.com>...<br>
> > If I am doing a calculation, and the result is<br>
> > <br>
> > 9.699999999999999<br>
> > <br>
> > when it should be exactly 9.7, what can I do to correct it?<br>
> > <br>
> > Thanks!<br>
>           <br>
> Welcome to the binary world! Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly. What you see there is about as close as it can get. To get some idea of the problem try representing 1/3 as an exact decimal fraction.<br>
> <br>
> Roger Stafford<br>
<br>
<br>
Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true). Any ideas?<br>
<br>
Thanks again.

Thu, 23 Jun 2011 05:15:05 +0000
Re: Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842527
Krishna Kumar
"Paul " <palex71@hotmail.com> wrote in message <itu0eh$9hb$1@newscl01ah.mathworks.com>...<br>
> If I am doing a calculation, and the result is<br>
> <br>
> 9.699999999999999<br>
> <br>
> when it should be exactly 9.7, what can I do to correct it?<br>
> <br>
> Thanks!<br>
Hi paul,<br>
You should round it off. Otherwise put the condition statement like this<br>
if abs(ab)<tol instead of if a==b,<br>
where tol is tolerance value, say, 10^5 etc. <br>
Or try this<br>
a1=sprintf('%0.1d',9.6999999999999);<br>
a2=sprintf('%0.1d',9.7)<br>
if strcmp(a1,a2).....<br>
<br>
HTH,<br>
Krishna

Thu, 23 Jun 2011 06:05:14 +0000
Re: Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842534
Roger Stafford
"Paul " <palex71@hotmail.com> wrote in message <itufrk$gc3$1@newscl01ah.mathworks.com>...<br>
> Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true). Any ideas?<br>
       <br>
There needs to be a careful distinction made between two concepts concerning a numerical result in matlab. There is the way the number is displayed to the user and there is the binary number actually stored in the computer's memory, and these are not necessarily the same. (What you see is not always exactly what you get.) It is only the stored number that is used for computations, with the displayed value being provided purely for the benefit of the user.<br>
<br>
In the case of a number entered as 9.7, the computer will actually store this binary number:<br>
<br>
1001.1011001100110011001100110011001100110011001100110<br>
<br>
It is the binary double precision's closest approximation to 9.7 and as you can see from the repeating pattern, no matter how many bits we were to provide the computer it would always be in error by some nonzero amount with the pattern continuing out indefinitely far.<br>
<br>
As for how such a number is displayed to the user, that all depends on how the display format is set. With 'format short' it appears as:<br>
<br>
9.7000<br>
<br>
With (my) 'format long' it is:<br>
<br>
9.70000000000000<br>
<br>
With fprintf('%18.16f\n',9.7) it is:<br>
<br>
9.6999999999999993<br>
<br>
With 'format hex' it is:<br>
<br>
4023666666666666<br>
<br>
This last, (in IEEE 754 hexadecimal format,) is the only one that gives the true value of the binary quantity that is stored.<br>
<br>
If as you stated earlier the number is the result of some computation, it is quite likely that it will have suffered a few bits worth of round off error at the low end, in which case the last three formats might be slightly different than above. In the case of the 'format short' if the correct answer is 9.7 this format would continue to give 9.7000 unless some very large errors had occurred in the computation. On the other hand, if the exact answer is supposed to be, say, 9.7000125, then the format short would continue to give you what would then be an erroneous 9.7000 .<br>
<br>
I hope that somewhere in the above meandering discussion there lies an answer to your question about how to "convert" the output to match another value  or rather, an explanation for why it would be almost impossible for such a conversion attempt to always succeed. I think you will have to give up the notion of always getting exact matches with numbers that have undergone different procedures and settle for only approximate matches. This is inherent in the very nature of numerical computations performed by digital computers.<br>
<br>
Roger Stafford

Thu, 23 Jun 2011 06:10:39 +0000
Re: Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842535
Rune Allnor
On Jun 23, 6:36 am, "Paul " <pale...@hotmail.com> wrote:<br>
> "Roger Stafford" wrote in message <itu0vd$as...@newscl01ah.mathworks.com>...<br>
> > "Paul " <pale...@hotmail.com> wrote in message <itu0eh$9h...@newscl01ah.mathworks.com>...<br>
> > > If I am doing a calculation, and the result is<br>
><br>
> > > 9.699999999999999<br>
><br>
> > > when it should be exactly 9.7, what can I do to correct it?<br>
><br>
> > > Thanks!<br>
> >           <br>
> > Welcome to the binary world! Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly. What you see there is about as close as it can get. To get some idea of the problem try representing 1/3 as an exact decimal fraction.<br>
><br>
> > Roger Stafford<br>
><br>
> Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true). Any ideas?<br>
<br>
Sure. Learn how to use insexact aritmetics:<br>
<br>
<a href="http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems">http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems</a><br>
<br>
Rune

Thu, 23 Jun 2011 22:08:04 +0000
Re: Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842688
Paul
Rune Allnor <allnor@tele.ntnu.no> wrote in message <0a83bac876a94786810f64eb0eee0eed@j23g2000yqc.googlegroups.com>...<br>
> On Jun 23, 6:36 am, "Paul " <pale...@hotmail.com> wrote:<br>
> > "Roger Stafford" wrote in message <itu0vd$as...@newscl01ah.mathworks.com>...<br>
> > > "Paul " <pale...@hotmail.com> wrote in message <itu0eh$9h...@newscl01ah.mathworks.com>...<br>
> > > > If I am doing a calculation, and the result is<br>
> ><br>
> > > > 9.699999999999999<br>
> ><br>
> > > > when it should be exactly 9.7, what can I do to correct it?<br>
> ><br>
> > > > Thanks!<br>
> > >           <br>
> > > Welcome to the binary world! Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly. What you see there is about as close as it can get. To get some idea of the problem try representing 1/3 as an exact decimal fraction.<br>
> ><br>
> > > Roger Stafford<br>
> ><br>
> > Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true). Any ideas?<br>
> <br>
> Sure. Learn how to use insexact aritmetics:<br>
> <br>
> <a href="http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems">http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems</a><br>
> <br>
> Rune<br>
<br>
Thanks everyone for the helpful replies. I did use if abs(xy)<tol.... and it appears to be working.<br>
<br>
Regards.

Mon, 27 Jun 2011 08:20:04 +0000
Re: Inexact output
http://www.mathworks.com/matlabcentral/newsreader/view_thread/309628#842983
gabriel villalobos
Besides binary finite representation, keep in mind that 9.699... with and infinite number of 9's is exactly equal to 9.7<br>
<br>
<br>
"Paul " <palex71@hotmail.com> wrote in message <iu0dg4$bo4$1@newscl01ah.mathworks.com>...<br>
> Rune Allnor <allnor@tele.ntnu.no> wrote in message <0a83bac876a94786810f64eb0eee0eed@j23g2000yqc.googlegroups.com>...<br>
> > On Jun 23, 6:36 am, "Paul " <pale...@hotmail.com> wrote:<br>
> > > "Roger Stafford" wrote in message <itu0vd$as...@newscl01ah.mathworks.com>...<br>
> > > > "Paul " <pale...@hotmail.com> wrote in message <itu0eh$9h...@newscl01ah.mathworks.com>...<br>
> > > > > If I am doing a calculation, and the result is<br>
> > ><br>
> > > > > 9.699999999999999<br>
> > ><br>
> > > > > when it should be exactly 9.7, what can I do to correct it?<br>
> > ><br>
> > > > > Thanks!<br>
> > > >           <br>
> > > > Welcome to the binary world! Your computer is using binary double precision floating point numbers and as such it cannot represent the decimal fraction 9.7 exactly. What you see there is about as close as it can get. To get some idea of the problem try representing 1/3 as an exact decimal fraction.<br>
> > ><br>
> > > > Roger Stafford<br>
> > ><br>
> > > Right... what I need is to convert the output to 9.7 so that it is equal to a 9.7 value read from another file (so conditional=true). Any ideas?<br>
> > <br>
> > Sure. Learn how to use insexact aritmetics:<br>
> > <br>
> > <a href="http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems">http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems</a><br>
> > <br>
> > Rune<br>
> <br>
> Thanks everyone for the helpful replies. I did use if abs(xy)<tol.... and it appears to be working.<br>
> <br>
> Regards.