Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
eps

Subject: eps

From: Pinpress

Date: 15 Mar, 2011 20:54:22

Message: 1 of 16

Hi,

I have a question regarding "eps" -- for double data type, the smallest positive number would be about: 2.225E-307. However, in Matlab, eps returns something around 1e-16, even on my 64-bit computer.

So why can't Matlab represent double data in better "granularity"? thanks.

Subject: eps

From: Steven_Lord

Date: 15 Mar, 2011 21:13:52

Message: 2 of 16



"Pinpress" <nothing@nothing.edu> wrote in message
news:ilojlu$esq$1@ginger.mathworks.com...
> Hi,
>
> I have a question regarding "eps" -- for double data type, the smallest
> positive number would be about: 2.225E-307.

REALMIN is the smallest positive normal number, yes. You can represent
numbers smaller than REALMIN, but those are denormal numbers.

> However, in Matlab, eps returns something around 1e-16, even on my 64-bit
> computer.

That is true. eps, or to be more explicit eps(1), is about 2.2e-16.

> So why can't Matlab represent double data in better "granularity"? thanks.

Because you're comparing apples (REALMIN) and oranges (EPS).

Read this for a description of how the IEEE double precision representation
works from Cleve:

http://www.mathworks.com/company/newsletters/news_notes/pdf/Fall96Cleve.pdf

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: eps

From: John D'Errico

Date: 15 Mar, 2011 21:30:19

Message: 3 of 16

"Pinpress" <nothing@nothing.edu> wrote in message <ilojlu$esq$1@ginger.mathworks.com>...
> Hi,
>
> I have a question regarding "eps" -- for double data type, the smallest positive number would be about: 2.225E-307. However, in Matlab, eps returns something around 1e-16, even on my 64-bit computer.
>
> So why can't Matlab represent double data in better "granularity"? thanks.

It CAN, and it CAN'T. You are mistaking two different
aspects of a number when written in floating point form.

For example, Matlab CAN represent the number

>> X = 1e-300;

In fact, matlab can represent numbers as small as realmin
in a double.

>> realmin
ans =
  2.2251e-308

This does not mean that MATLAB stores 300 digits of
precision though. It still stores only 52 bits of precision
in that double, so roughly 16 decimal digits. We see that
information in eps(1), a function that yields the smallest
number that can be added to 1 and yield a result distinct
from 1.

>> eps(1)
ans =
   2.2204e-16

Thus, perform the test:

>> (1 + eps(1)) == 1
ans =
     0

You see that the two numbers are distinct. However, try
it with a slightly smaller number.

>> (1 + eps(1)/2) == 1
ans =
     1

They are not distinct. This reflects the 52 bits of precision
in a double. Look at it another way:

>> log2(eps(1))
ans =
   -52

There is a difference between eps and the dynamic range
of the numbers that can be stored in a double.

John

Subject: eps

From: Pinpress

Date: 15 Mar, 2011 21:37:06

Message: 4 of 16

Thanks Steve for the pointer. The reason I am asking about this is that I just realize that when I was trying to solve a linear system of equations using Matlab, it always says that the matrix is almost singular, while if I use my other Fortran code (using SPLIB), it solves the equations fine, within only 10 iterations. I think this is probably in Matlab (and true for any codes written in Matlab, perhaps?), there is the "eps" that controls the roundoff error, which makes it infeasible for these types of solutions.

Any further rectification to solve the equations in Matlab?? Or I have to resort to other code?? Thanks.


>
> Because you're comparing apples (REALMIN) and oranges (EPS).
>
> Read this for a description of how the IEEE double precision representation
> works from Cleve:
>
> http://www.mathworks.com/company/newsletters/news_notes/pdf/Fall96Cleve.pdf
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

Subject: eps

From: Think two, count blue.

Date: 15 Mar, 2011 22:15:22

Message: 5 of 16

On 11-03-15 03:54 PM, Pinpress wrote:

> I have a question regarding "eps" -- for double data type, the smallest
> positive number would be about: 2.225E-307. However, in Matlab, eps returns
> something around 1e-16, even on my 64-bit computer.
>
> So why can't Matlab represent double data in better "granularity"? thanks.

Are you forgetting the argument?

 >> eps(realmin)
ans =
      4.94065645841247e-324


eps is not an absolute quantity: it is quantity that applies to a small range
of numbers

Subject: eps

From: Bruno Luong

Date: 15 Mar, 2011 22:42:06

Message: 6 of 16

"Think two, count blue." <roberson@hushmail.com> wrote in message <iloods$4dv$1@nrc-news.nrc.ca>...

>
> >> eps(realmin)
> ans =
> 4.94065645841247e-324

Humm you are quite distubing with your posts today Walter.

>> eps(eps(realmin))

ans =

  4.9407e-324

>> eps(eps(realmin))<realmin

ans =

     1

So 0 < eps(eps(realmin)) == eps(realmin) < realmin

I'm confused. What exactly is realmin means?

Bruno

Subject: eps

From: Steven_Lord

Date: 16 Mar, 2011 13:21:29

Message: 7 of 16



"Pinpress" <nothing@nothing.edu> wrote in message
news:ilom62$lsl$1@ginger.mathworks.com...
> Thanks Steve for the pointer. The reason I am asking about this is that I
> just realize that when I was trying to solve a linear system of equations
> using Matlab, it always says that the matrix is almost singular, while if
> I use my other Fortran code (using SPLIB), it solves the equations fine,
> within only 10 iterations. I think this is probably in Matlab (and true
> for any codes written in Matlab, perhaps?), there is the "eps" that
> controls the roundoff error, which makes it infeasible for these types of
> solutions.

What is the condition number of your coefficient matrix? You can find this
using COND or CONDEST.

It's possible that the threshold for ill-conditioning used by the Fortran
solver is higher than the threshold beyond which MATLAB decides to warn, and
your matrix has a condition number that falls between the two.

> Any further rectification to solve the equations in Matlab?? Or I have to
> resort to other code?? Thanks.

It depends. What do the equations represent? What problem are you trying to
solve?

Another reference, also by Cleve, that may be interesting to you is section
2.9 of the Linear Equations section of "Numerical Computing with MATLAB"

http://www.mathworks.com/moler/chapters.html

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: eps

From: Steven_Lord

Date: 16 Mar, 2011 13:31:26

Message: 8 of 16



"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
news:ilopvu$282$1@ginger.mathworks.com...
> "Think two, count blue." <roberson@hushmail.com> wrote in message
> <iloods$4dv$1@nrc-news.nrc.ca>...
>
>>
>> >> eps(realmin)
>> ans =
>> 4.94065645841247e-324
>
> Humm you are quite distubing with your posts today Walter.
>
>>> eps(eps(realmin))
>
> ans =
>
> 4.9407e-324
>
>>> eps(eps(realmin))<realmin
>
> ans =
>
> 1
>
> So 0 < eps(eps(realmin)) == eps(realmin) < realmin
>
> I'm confused. What exactly is realmin means?

As I said in my original message, REALMIN (with no inputs or with the string
'double' as input) returns the smallest positive normal number. It is
possible to represent smaller positive numbers called denormal numbers.

http://en.wikipedia.org/wiki/Denormal

If you display REALMIN using FORMAT HEX, you'll note that the third hex
digit is 1. That means the (biased) exponent of the number is 1, and the
number is normal. If you display eps(realmin) that same way, the exponent is
all zero indicating it's denormal.

http://en.wikipedia.org/wiki/IEEE_754-1985#Cases

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: eps

From: Pinpress

Date: 16 Mar, 2011 13:57:04

Message: 9 of 16

The condition number is on the order of 1e-19 in matlab, indeed smaller than eps.

Just curious why Matlab decides to have such a double number "granularity", or this is actually based on an old IEEE standard. The same equation I am solving using Fortran, it only takes ~10 iterations to converge to an acceptable solution, but this is not solvable in Matlab.


>
> What is the condition number of your coefficient matrix? You can find this
> using COND or CONDEST.
>
> It's possible that the threshold for ill-conditioning used by the Fortran
> solver is higher than the threshold beyond which MATLAB decides to warn, and
> your matrix has a condition number that falls between the two.
>
> > Any further rectification to solve the equations in Matlab?? Or I have to
> > resort to other code?? Thanks.
>
> It depends. What do the equations represent? What problem are you trying to
> solve?
>
> Another reference, also by Cleve, that may be interesting to you is section
> 2.9 of the Linear Equations section of "Numerical Computing with MATLAB"
>
> http://www.mathworks.com/moler/chapters.html
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

Subject: eps

From: Nasser M. Abbasi

Date: 16 Mar, 2011 14:09:22

Message: 10 of 16

On 3/16/2011 6:57 AM, Pinpress wrote:

> The condition number is on the order of 1e-19 in matlab, indeed smaller than eps.

huh?

How can condition number of a matrix be less than 1?

condition number is ratio of largest to smallest eigenvalue (in absolute values).

So, the smallest it can be is 1.


--Nasser

Subject: eps

From: Steven_Lord

Date: 16 Mar, 2011 14:17:00

Message: 11 of 16



"Pinpress" <nospam__@yahoo.com> wrote in message
news:ilqfjg$ig2$1@ginger.mathworks.com...
> The condition number is on the order of 1e-19 in matlab, indeed smaller
> than eps.

That's the reciprocal condition number; condition numbers must be greater
than or equal to 1. My guess is that COND gives you something on the order
of 1e18 to 1e20. And yes, that is a very ill-conditioned problem. Again,
what specific problem are you trying to solve? Are you trying to perform
regression using the normal equations or a Vandermonde matrix? If so, there
may be techniques you can use that avoid such an ill-conditioned coefficient
matrix.

> Just curious why Matlab decides to have such a double number
> "granularity", or this is actually based on an old IEEE standard.

IEEE 754, revised most recently in 2008. And it's not just MATLAB that uses
this storage format.

> The same equation I am solving using Fortran, it only takes ~10 iterations
> to converge to an acceptable solution, but this is not solvable in Matlab.

With such a badly conditioned matrix, I strongly urge you to check your
Fortran solution to make sure it's not completely wrong. I know that's a
strong statement, but with a condition number that large I think it's
justified. [You should ALWAYS check the answer on general principles, but
it's particularly important in this particular scenario.]

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: eps

From: Pinpress

Date: 16 Mar, 2011 15:36:05

Message: 12 of 16

Thanks all.

Matlab reports that RCND is on the order of 1e-19 to 1e-20.

I am trying to solve a finite element problem. I have traced another problem related to Matlab EPS, or the "granularity" of double numbers. Supposedly that the stiffness matrix should be symmetric. However, I observed that for two corresponding element, K(i,j) and K(j,i), they differ:

31666.66666666668200000000000000000000000000000000000000
31666.66666666669000000000000000000000000000000000000000

This small change is enough to cause the solution change. In Matlab, if I use:

u = K\b;

It gives wrong answer. Interestingly, if I use:

u = K'\b;

It returns "correct" answer, which can be verified by comparing K'*u and b -- they are very close.

(Another note, if I replace K or K' by (K+K')/2, Matlab would give me yet another solution, but is also wrong.)

Using the same data, if I use my Fortran code, using either K or K' return the same answer.

Anyone can help solve the mystery?? Or is this the time Matlab could consider fine-tune the double number "granularity"??

Subject: eps

From: Bruno Luong

Date: 16 Mar, 2011 16:31:05

Message: 13 of 16

"Pinpress" wrote in message <ilqld5$946$1@ginger.mathworks.com>...

>
> Anyone can help solve the mystery?? Or is this the time Matlab could consider fine-tune the double number "granularity"??

There is no mystery, the condition number give the sensitivity of the solution with respect to the matrix and forcing. This is well known in numerical analysis and linear algebra.

The question is how you get the assembled matrix with such bad condition number. If the PDE is well-posed and the boundary condition is correctly take into account, the matrix would be better conditioned (é 10^3 in my expecrience). Please check your matrix and formulation.

Bruno

Subject: eps

From: Pinpress

Date: 16 Mar, 2011 16:46:03

Message: 14 of 16

Hi Bruno,

My system is actually a coupled system -- solid and fluid. Terms relating the solid and those w.r.t. fluid differ quite a lot, that's why I have a very bad condition number.

I guess I wonder why if I transpose the matrix before solution, it works fine in Matlab. However, the Fortran code is apparently not (or less?) sensitive to this.

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <ilqok9$ioj$1@ginger.mathworks.com>...
> "Pinpress" wrote in message <ilqld5$946$1@ginger.mathworks.com>...
>
> >
> > Anyone can help solve the mystery?? Or is this the time Matlab could consider fine-tune the double number "granularity"??
>
> There is no mystery, the condition number give the sensitivity of the solution with respect to the matrix and forcing. This is well known in numerical analysis and linear algebra.
>
> The question is how you get the assembled matrix with such bad condition number. If the PDE is well-posed and the boundary condition is correctly take into account, the matrix would be better conditioned (é 10^3 in my expecrience). Please check your matrix and formulation.
>
> Bruno

Subject: eps

From: Steven_Lord

Date: 16 Mar, 2011 17:41:39

Message: 15 of 16



"Pinpress" <nospam__@yahoo.com> wrote in message
news:ilqld5$946$1@ginger.mathworks.com...
> Thanks all.
>
> Matlab reports that RCND is on the order of 1e-19 to 1e-20.
>
> I am trying to solve a finite element problem. I have traced another
> problem related to Matlab EPS, or the "granularity" of double numbers.

You keep on using the term "granularity" like there is only one level of
granularity. That is incorrect.

The spacing between adjacent representable numbers is NOT constant; numbers
are more closely spaced when the numbers are small than when they are large.
Compare:

eps(1)
eps(10)
eps(2^52)
eps(2^53)

> Supposedly that the stiffness matrix should be symmetric. However, I
> observed that for two corresponding element, K(i,j) and K(j,i), they
> differ:
>
> 31666.66666666668200000000000000000000000000000000000000
> 31666.66666666669000000000000000000000000000000000000000
>
> This small change is enough to cause the solution change.

That's correct. If you reread section 2.9 in the chapter from Cleve's book
to which I directed you, particularly page 17, you'll see this quote:

"This shows that the condition number is a relative error magnification
factor.
Changes in the right-hand side can cause changes K(A) times as large in the
solution.
It turns out that the same is true of changes in the coefficient matrix
itself."

So the difference in the coefficient (on the order of 1e-11) when multiplied
by the condition number could cause a change of about 1e8 in the solution!

> In Matlab, if I use:
>
> u = K\b;
>
> It gives wrong answer. Interestingly, if I use:
>
> u = K'\b;
>
> It returns "correct" answer, which can be verified by comparing K'*u and
> b -- they are very close.
>
> (Another note, if I replace K or K' by (K+K')/2, Matlab would give me yet
> another solution, but is also wrong.)
>
> Using the same data, if I use my Fortran code, using either K or K' return
> the same answer.
> Anyone can help solve the mystery?? Or is this the time Matlab could
> consider fine-tune the double number "granularity"??

In my opinion, that is not the correct solution even if it were something we
could do. Figuring out why your matrix has such a high condition number and
if there's a way to reduce it is the correct solution, again in my opinion.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: eps

From: Bruno Luong

Date: 16 Mar, 2011 18:46:05

Message: 16 of 16

"Pinpress" wrote in message <ilqpgb$lbc$1@ginger.mathworks.com>...
> Hi Bruno,
>
> My system is actually a coupled system -- solid and fluid. Terms relating the solid and those w.r.t. fluid differ quite a lot, that's why I have a very bad condition number.
>
> I guess I wonder why if I transpose the matrix before solution, it works fine in Matlab. However, the Fortran code is apparently not (or less?) sensitive to this.

If the bad scaling is due to large different scaling, then a simple rescale will reduce the condition number. Work with normalized unit instead.

Read the literature on condition number, preconditioning, etc... No numerical engineers can afford not to know those notions.

Bruno

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us