MATLAB Central Newsreader - singular covariance matrix and svd

singular covariance matrix and svd

francesco santi — Tue, 17 Feb 2009 10:33:01 -0500

Hello to everybody.
Here is my problem:
I have to compute a covariance matrix S at each step of my algorithm.
This is done over an image matrix.
The image matrix values are normalized to [0,1].
Problems arise when I have to compute the determinant of S or its inverse.
Sometimes it happens that S is singular or close to singular so I get numerical instability. Should it happen, all the formulas following the computation of S will have a Nan or Inf value and the algorithm would not converge.
I've found out the svd Matlab built-in function and I was wondering if it might be helpful to solve my probem and how.

The general solution is to analyzie the covariance matrix before passing it to other formulas and modify the values that make it singular.
The question is: how?

Any help would be appreciated.
Thanks in advance.

Re: singular covariance matrix and svd

Rune Allnor — Tue, 17 Feb 2009 11:58:47 -0500

On 17 Feb, 11:33, "francesco santi" <fpsa...@gmail.com> wrote:
> Hello to everybody.
> Here is my problem:
> I have to compute a covariance matrix S at each step of my algorithm.
> This is done over an image matrix.
> The image matrix values are normalized to [0,1].
> Problems arise when I have to compute the determinant of S or its inverse.
> Sometimes it happens that S is singular or close to singular so I get numerical instability.

If the matrix is singular, the determinant is 0, or
nearly 0. T

> Should it happen, all the formulas following the computation of S will have a Nan or Inf value and the algorithm would not converge.

In that case you either work with an ill-conditioned problem,
use the wrong algorithm, or use an unstable implementation.
Or all of the above.

> I've found out the svd Matlab built-in function and I was wondering if it might be helpful to solve my probem and how.

The SVD *might* be helpful for certain types of problems,
but not all. Learning what the SVD is, how it works and
how to use it is far beyond USENET, the ideal approach
is to take a class on linear algebra.

If that's not an option, find a copy of Strang's "Linear
Algebra and its Applications" and read.

Rune

Re: singular covariance matrix and svd

Roger Stafford — Tue, 17 Feb 2009 16:25:04 -0500

"francesco santi" <fpsanti@gmail.com> wrote in message <gne3ot$cs9$1@fred.mathworks.com>...
> ......
> The general solution is to analyzie the covariance matrix before passing it to other formulas and modify the values that make it singular.
> ......

Francesco, it would be more appropriate in my opinion to determine the eigenvalues of S rather than using singular value decomposition. For covariance matrices the eigenvalues must all be real and non-negative. They become singular when any of these eigenvalues is zero. If you artificially alter the singular values from 'svd', you could end up with a matrix that is no longer a valid covariance matrix - that is, no longer positive semi-definite - but if all eigenvalues are kept positive this can't happen.

However, the idea of artificially altering a covariance matrix simply to make some numerical procedure work seems contrived to me. Perhaps you should take Rune's advice and consider a change in algorithm for handling singular covariance situations. As you undoubtedly are aware, such an occurrence indicates some linear relationship strictly holds among the variables you are working with. At least one of them can be exactly predicted from others. You really ought to be able to handle such relationships in the way you process your data other than resorting to artifices such as changing covariance matrices. Otherwise you may draw unwarranted conclusions from your analysis.

Roger Stafford