Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
negative eigenvalue in principal component analysis

Subject: negative eigenvalue in principal component analysis

From: aymer

Date: 2 Mar, 2012 18:44:30

Message: 1 of 5

Hello there,

I am trying to reconstruct a function using PCA. Here is what I do.
I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.

The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???

can someone kindly suggest a solution or some references...

thanx in advance

Subject: negative eigenvalue in principal component analysis

From: Greg Heath

Date: 4 Mar, 2012 20:34:14

Message: 2 of 5

"aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> Hello there,
>
> I am trying to reconstruct a function using PCA. Here is what I do.
> I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
>
> The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
>
> can someone kindly suggest a solution or some references...
> thanx in advance

Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.

Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.

Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
magnitude values of the correlation coefficient matrix.
 
Hope this helps.

Greg

Subject: negative eigenvalue in principal component analysis

From: aymer

Date: 5 Mar, 2012 18:59:13

Message: 3 of 5

Hello Greg,

Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.

Any idea where I might be making a mistake??
thank you for your time..

"Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...
> "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> > Hello there,
> >
> > I am trying to reconstruct a function using PCA. Here is what I do.
> > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
> >
> > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
> >
> > can someone kindly suggest a solution or some references...
> > thanx in advance
>
> Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.
>
> Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.
>
> Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
> magnitude values of the correlation coefficient matrix.
>
> Hope this helps.
>
> Greg

Subject: negative eigenvalue in principal component analysis

From: Greg Heath

Date: 6 Mar, 2012 23:45:11

Message: 4 of 5

"aymer " <remyansonu@gmail.com> wrote in message <jj32e1$ovd$1@newscl01ah.mathworks.com>...
> Hello Greg,
>
> Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.
>
> Any idea where I might be making a mistake??
> thank you for your time..
>
> "Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...
> > "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> > > Hello there,
> > >
> > > I am trying to reconstruct a function using PCA. Here is what I do.
> > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
> > >
> > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
> > >
> > > can someone kindly suggest a solution or some references...
> > > thanx in advance
> >
> > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.
> >
> > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.
> >
> > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
> > magnitude values of the correlation coefficient matrix.
> >
> > Hope this helps.
> >
> > Greg

PLEASE DO NOT TOP POST!
PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE.

Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher.

Greg

Subject: negative eigenvalue in principal component analysis

From: aymer

Date: 8 Mar, 2012 17:53:34

Message: 5 of 5

"Greg Heath" <heath@alumni.brown.edu> wrote in message <jj67i7$i59$1@newscl01ah.mathworks.com>...
> "aymer " <remyansonu@gmail.com> wrote in message <jj32e1$ovd$1@newscl01ah.mathworks.com>...
> > Hello Greg,
> >
> > Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.
> >
> > Any idea where I might be making a mistake??
> > thank you for your time..
> >
> > "Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...
> > > "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> > > > Hello there,
> > > >
> > > > I am trying to reconstruct a function using PCA. Here is what I do.
> > > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
> > > >
> > > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
> > > >
> > > > can someone kindly suggest a solution or some references...
> > > > thanx in advance
> > >
> > > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.
> > >
> > > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.
> > >
> > > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
> > > magnitude values of the correlation coefficient matrix.
> > >
> > > Hope this helps.
> > >
> > > Greg
>
> PLEASE DO NOT TOP POST!
> PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE.
>
> Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher.
>
> Greg

I'll keep that in mind Greg...sorry about that

thanx for your time

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us