From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: negative eigenvalue in principal component analysis
Date: Mon, 5 Mar 2012 18:59:13 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 29
Message-ID: <jj32e1$ovd$>
References: <jir4ee$t1d$> <jj0jk6$ntt$>
Reply-To: <HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: 1330973953 25581 (5 Mar 2012 18:59:13 GMT)
NNTP-Posting-Date: Mon, 5 Mar 2012 18:59:13 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 3285432
Xref: comp.soft-sys.matlab:759955

Hello Greg,

Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.

Any idea where I might be making a mistake??
thank you for your time..

"Greg Heath" <> wrote in message <jj0jk6$ntt$>...
> "aymer" wrote in message <jir4ee$t1d$>...
> > Hello there,
> > 
> > I am trying to reconstruct a function using PCA. Here is what I do.
> > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
> > 
> > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
> > 
> > can someone kindly suggest a solution or some references...
> > thanx in advance
> Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.
> Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. 
> Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher 
> magnitude values of the correlation coefficient matrix.
> Hope this helps.
> Greg