# How to drop last N round-off error digits?

16 views (last 30 days)
Ilya on 17 Jun 2014
a=sqrt(pi) returns 1.77245385090552, where the last digit is not reliable, and this digit is of the order 1e-14.
Now, a=sqrt(53512343*pi) returns 12965.8776658265, where the last digit or two are not reliable, and these digits are of the order 1e-9 -- 1e-10.
The question is how to get rid of unreliable digits in a standardized and well-proven manner?
The problem is that these last digits may deviate strongly depending on input data of an algorithm leading to situations when some results that should be in fact equal are not equal in the last digit.
Note that
format long
should be executed before typing any expressions in order to see the full number of digits of the results.
Note also that "comparison up to some precision" is not a solution, as I need to store only one unique value of the answer, thus neglecting all its variations due to round-off errors. And I'm not aware about any unique()-like function that works with some tolerance.
The round-off errors in my application are usually small (i.e. they are in the last 1-2 digits).

Titus Edelhofer on 17 Jun 2014
Hi Ilya,
if you are happy with an output, then use fprintf:
a=sqrt(pi);
fprintf('a = %.14g\n', a)
a = 1.7724538509055
Otherwise: use e.g. eps to find the roundoff digit.
a = sqrt(pi);
e = eps(a);
% how many decimal digits it this?
digits = floor(-log10(e));
% cut off the last two
digits = digits - 2;
% round to this digit
aRound = round(a*10^digits) / (10^digits)
aRound =
1.772453850905500
Titus

Ilya on 17 Jun 2014
Of course, I need the second option =) And it looks pretty nice.. However, I wonder if there's some standardized solution since the problem seems to be quite common.
Srighakollapu Vali M on 21 Mar 2018
This works fine for non zero values. If the variable takes 0 instead of 'pi' or some nonzero number, after running this code, I am getting NaN.How to deal the above problem if the value is zero especially, if we are working with matrices where some entries may be zero.

John D'Errico on 17 Jun 2014
The simple scale/round/unscale is the standard answer. HOWEVER, don't expect it to help you completely, because you are working in floating point arithmetic. For example...
X0 = 1.2345678901234567
X0 =
1.23456789012346
Lets try to zero out the last few digits.
X1 = round(1e12*X0)/1e12
X1 =
1.234567890123
While it appears that this worked, did it? I'll use my HPF tool as a simple way of extracting the exact number as MATLAB stores it. Remember that MATLAB uses a binary form,so while it might print out in decimal form, and LOOK like it is what you think, the true value as stored is not so exact, EVEN AFTER ROUNDING.
hpf(X1,55)
ans =
1.234567890122999944679804684710688889026641845703125000
Finally, while format long HELPS, it does not truly show the effect of those 53 bits that MATLAB carries in the binary form.

#### 1 Comment

Ilya on 17 Jun 2014
Thanks a lot! I simply received several very elegant and unexpected solutions on this forum and hoped to receive another such solution. Thanks also for the HPF tool.

Albert on 12 Oct 2017
floor(1e12*X0)/1e12
this may work better than
round(1e12*X0)/1e12