http://www.mathworks.com/matlabcentral/newsreader/view_thread/317549
MATLAB Central Newsreader  negative eigenvalue in principal component analysis
Feed for thread: negative eigenvalue in principal component analysis
enus
©19942015 by MathWorks, Inc.
webmaster@mathworks.com
MATLAB Central Newsreader
http://blogs.law.harvard.edu/tech/rss
60
MathWorks
http://www.mathworks.com/images/membrane_icon.gif

Fri, 02 Mar 2012 18:44:30 +0000
negative eigenvalue in principal component analysis
http://www.mathworks.com/matlabcentral/newsreader/view_thread/317549#868666
aymer
Hello there,<br>
<br>
I am trying to reconstruct a function using PCA. Here is what I do.<br>
I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chisquared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.<br>
<br>
The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???<br>
<br>
can someone kindly suggest a solution or some references...<br>
<br>
thanx in advance

Sun, 04 Mar 2012 20:34:14 +0000
Re: negative eigenvalue in principal component analysis
http://www.mathworks.com/matlabcentral/newsreader/view_thread/317549#868832
Greg Heath
"aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...<br>
> Hello there,<br>
> <br>
> I am trying to reconstruct a function using PCA. Here is what I do.<br>
> I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chisquared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.<br>
> <br>
> The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???<br>
> <br>
> can someone kindly suggest a solution or some references...<br>
> thanx in advance<br>
<br>
Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.<br>
<br>
Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. <br>
<br>
Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher <br>
magnitude values of the correlation coefficient matrix.<br>
<br>
Hope this helps.<br>
<br>
Greg

Mon, 05 Mar 2012 18:59:13 +0000
Re: negative eigenvalue in principal component analysis
http://www.mathworks.com/matlabcentral/newsreader/view_thread/317549#868975
aymer
Hello Greg,<br>
<br>
Thank you for you reply. I generate my chisquare function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.<br>
<br>
Any idea where I might be making a mistake??<br>
thank you for your time..<br>
<br>
"Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...<br>
> "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...<br>
> > Hello there,<br>
> > <br>
> > I am trying to reconstruct a function using PCA. Here is what I do.<br>
> > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chisquared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.<br>
> > <br>
> > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???<br>
> > <br>
> > can someone kindly suggest a solution or some references...<br>
> > thanx in advance<br>
> <br>
> Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.<br>
> <br>
> Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. <br>
> <br>
> Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher <br>
> magnitude values of the correlation coefficient matrix.<br>
> <br>
> Hope this helps.<br>
> <br>
> Greg

Tue, 06 Mar 2012 23:45:11 +0000
Re: negative eigenvalue in principal component analysis
http://www.mathworks.com/matlabcentral/newsreader/view_thread/317549#869149
Greg Heath
"aymer " <remyansonu@gmail.com> wrote in message <jj32e1$ovd$1@newscl01ah.mathworks.com>...<br>
> Hello Greg,<br>
> <br>
> Thank you for you reply. I generate my chisquare function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.<br>
> <br>
> Any idea where I might be making a mistake??<br>
> thank you for your time..<br>
> <br>
> "Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...<br>
> > "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...<br>
> > > Hello there,<br>
> > > <br>
> > > I am trying to reconstruct a function using PCA. Here is what I do.<br>
> > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chisquared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.<br>
> > > <br>
> > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???<br>
> > > <br>
> > > can someone kindly suggest a solution or some references...<br>
> > > thanx in advance<br>
> > <br>
> > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.<br>
> > <br>
> > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. <br>
> > <br>
> > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher <br>
> > magnitude values of the correlation coefficient matrix.<br>
> > <br>
> > Hope this helps.<br>
> > <br>
> > Greg<br>
<br>
PLEASE DO NOT TOP POST!<br>
PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE.<br>
<br>
Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher.<br>
<br>
Greg

Thu, 08 Mar 2012 17:53:34 +0000
Re: negative eigenvalue in principal component analysis
http://www.mathworks.com/matlabcentral/newsreader/view_thread/317549#869409
aymer
"Greg Heath" <heath@alumni.brown.edu> wrote in message <jj67i7$i59$1@newscl01ah.mathworks.com>...<br>
> "aymer " <remyansonu@gmail.com> wrote in message <jj32e1$ovd$1@newscl01ah.mathworks.com>...<br>
> > Hello Greg,<br>
> > <br>
> > Thank you for you reply. I generate my chisquare function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.<br>
> > <br>
> > Any idea where I might be making a mistake??<br>
> > thank you for your time..<br>
> > <br>
> > "Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...<br>
> > > "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...<br>
> > > > Hello there,<br>
> > > > <br>
> > > > I am trying to reconstruct a function using PCA. Here is what I do.<br>
> > > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chisquared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.<br>
> > > > <br>
> > > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???<br>
> > > > <br>
> > > > can someone kindly suggest a solution or some references...<br>
> > > > thanx in advance<br>
> > > <br>
> > > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.<br>
> > > <br>
> > > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. <br>
> > > <br>
> > > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher <br>
> > > magnitude values of the correlation coefficient matrix.<br>
> > > <br>
> > > Hope this helps.<br>
> > > <br>
> > > Greg<br>
> <br>
> PLEASE DO NOT TOP POST!<br>
> PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE.<br>
> <br>
> Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher.<br>
> <br>
> Greg<br>
<br>
I'll keep that in mind Greg...sorry about that<br>
<br>
thanx for your time