How to drop last N round-off error digits?

9 views (last 30 days)
a=sqrt(pi) returns 1.77245385090552, where the last digit is not reliable, and this digit is of the order 1e-14.
Now, a=sqrt(53512343*pi) returns 12965.8776658265, where the last digit or two are not reliable, and these digits are of the order 1e-9 -- 1e-10.
The question is how to get rid of unreliable digits in a standardized and well-proven manner?
The problem is that these last digits may deviate strongly depending on input data of an algorithm leading to situations when some results that should be in fact equal are not equal in the last digit.
Note that
format long
should be executed before typing any expressions in order to see the full number of digits of the results.
Note also that "comparison up to some precision" is not a solution, as I need to store only one unique value of the answer, thus neglecting all its variations due to round-off errors. And I'm not aware about any unique()-like function that works with some tolerance.
The round-off errors in my application are usually small (i.e. they are in the last 1-2 digits).

Accepted Answer

Titus Edelhofer
Titus Edelhofer on 17 Jun 2014
Hi Ilya,
if you are happy with an output, then use fprintf:
fprintf('a = %.14g\n', a)
a = 1.7724538509055
Otherwise: use e.g. eps to find the roundoff digit.
a = sqrt(pi);
e = eps(a);
% how many decimal digits it this?
digits = floor(-log10(e));
% cut off the last two
digits = digits - 2;
% round to this digit
aRound = round(a*10^digits) / (10^digits)
aRound =
Srighakollapu Vali M
Srighakollapu Vali M on 21 Mar 2018
This works fine for non zero values. If the variable takes 0 instead of 'pi' or some nonzero number, after running this code, I am getting NaN.How to deal the above problem if the value is zero especially, if we are working with matrices where some entries may be zero.

Sign in to comment.

More Answers (2)

John D'Errico
John D'Errico on 17 Jun 2014
The simple scale/round/unscale is the standard answer. HOWEVER, don't expect it to help you completely, because you are working in floating point arithmetic. For example...
X0 = 1.2345678901234567
X0 =
Lets try to zero out the last few digits.
X1 = round(1e12*X0)/1e12
X1 =
While it appears that this worked, did it? I'll use my HPF tool as a simple way of extracting the exact number as MATLAB stores it. Remember that MATLAB uses a binary form,so while it might print out in decimal form, and LOOK like it is what you think, the true value as stored is not so exact, EVEN AFTER ROUNDING.
ans =
Finally, while format long HELPS, it does not truly show the effect of those 53 bits that MATLAB carries in the binary form.
  1 Comment
Ilya on 17 Jun 2014
Thanks a lot! I simply received several very elegant and unexpected solutions on this forum and hoped to receive another such solution. Thanks also for the HPF tool.

Sign in to comment.

Albert on 12 Oct 2017
this may work better than

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!