Thread Subject: Large eigenvector decompositions

Hi,


Does anybody know if there's an alternative to 'eig' which is more
memory efficient. I need an eigenvector decomposition of a
(2000x10000) matrix.


Thanks,


Gareth

Gareth Edwards wrote:


> Does anybody know if there's an alternative to 'eig' which is more
> memory efficient. I need an eigenvector decomposition of a
> (2000x10000) matrix.

If you want just the first (or last) few eigenvectors of the matrix, try
svds

--
Paige Miller
paige.miller@kodak.com
http://www.kodak.com

"It's nothing until I call it!" -- Bill Klem, NL Umpire
"When you get the choice to sit it out or dance, I hope you dance" --
Lee Ann Womack

In article <eeb11e6.-1@WebX.raydaftYaTP>, "Gareth Edwards"
<gareth.edwards@image-metrics.com> wrote:

> Does anybody know if there's an alternative to 'eig' which is more
> memory efficient. I need an eigenvector decomposition of a
> (2000x10000) matrix.

Thats interesting. I thought eigenvectors only
applied to square matrices. Is this a new
feature of matlab?

John

--

I found that the routine bigpca.m from the PLS toolbox is excellent for this
type of job. It doesn't need to compute the covariance matrix (or the
inverse) of all your data matrix, just the first n components you want.
http://www.eigenvector.com/PLS_Toolbox.html

HTH

Hugo


"Gareth Edwards" <gareth.edwards@image-metrics.com> wrote in message
news:eeb11e6.-1@WebX.raydaftYaTP...
> Hi,
>
>
> Does anybody know if there's an alternative to 'eig' which is more
> memory efficient. I need an eigenvector decomposition of a
> (2000x10000) matrix.
>
>
> Thanks,
>
>
> Gareth

John is right of course - I should have said I'm doing PCA on
2000x10000 data matrix. Anyway, it's too big for 'eig' or 'svd' even
though it can be mades a 2000x2000 problem.


John D'Errico wrote:
>
>
> In article <eeb11e6.-1@WebX.raydaftYaTP>, "Gareth Edwards"
> <gareth.edwards@image-metrics.com> wrote:
>
>> Does anybody know if there's an alternative to 'eig' which is
more
>> memory efficient. I need an eigenvector decomposition of a
>> (2000x10000) matrix.
>
> Thats interesting. I thought eigenvectors only
> applied to square matrices. Is this a new
> feature of matlab?
>
> John
>
> --
>
>

John is right of course - I should have said I'm doing PCA on
2000x10000 data matrix. Anyway, it's too big for 'eig' or 'svd' even
though it can be mades a 2000x2000 problem.


John D'Errico wrote:
>
>
> In article <eeb11e6.-1@WebX.raydaftYaTP>, "Gareth Edwards"
> <gareth.edwards@image-metrics.com> wrote:
>
>> Does anybody know if there's an alternative to 'eig' which is
more
>> memory efficient. I need an eigenvector decomposition of a
>> (2000x10000) matrix.
>
> Thats interesting. I thought eigenvectors only
> applied to square matrices. Is this a new
> feature of matlab?
>
> John
>
> --
>
>

In article <eeb11e6.3@WebX.raydaftYaTP>, "Gareth Edwards"
<gareth.edwards@image-metrics.com> wrote:

> John is right of course - I should have said I'm doing PCA on
> 2000x10000 data matrix. Anyway, it's too big for 'eig' or 'svd' even
> though it can be mades a 2000x2000 problem.
>

Yeah, I kind of figured it was pca. Eigs, or svds
if you prefer, will give you only a few eigenvalues.

Even the power method will give you the first few
eigenvalue/eigenvector pairs. Use deflation after
you get the first eigenvector to kill off the
corresponding eigenvalue, then repeat.

John

--

On Tue, 27 Aug 2002 07:29:22 -0400, "Gareth Edwards"
<gareth.edwards@image-metrics.com> wrote:

>John is right of course - I should have said I'm doing PCA on
>2000x10000 data matrix. Anyway, it's too big for 'eig' or 'svd' even
>though it can be mades a 2000x2000 problem.

Many PCA routines assume that the number of rows is larger than the
number of columns. For small problems it doesn't matter - for large
problems it matters a lot.

Fortunately, you can simply perform the PCA on X' and transform the
result into the PCA you would have obtained from doing it on X.

  Lars

Lars Gregersen
www.rndee.dk
lg@rndee.dk

lg@rndee.dk (Lars Gregersen) wrote in message <3d6fc2c6.11637794@news.cybercity.dk>...
> On Tue, 27 Aug 2002 07:29:22 -0400, "Gareth Edwards"
> <gareth.edwards@image-metrics.com> wrote:
>
> >John is right of course - I should have said I'm doing PCA on
> >2000x10000 data matrix. Anyway, it's too big for 'eig' or 'svd' even
> >though it can be mades a 2000x2000 problem.
>
> Many PCA routines assume that the number of rows is larger than the
> number of columns. For small problems it doesn't matter - for large
> problems it matters a lot.
>
> Fortunately, you can simply perform the PCA on X' and transform the
> result into the PCA you would have obtained from doing it on X.
>
> Lars
>
> Lars Gregersen
> www.rndee.dk
> lg@rndee.dk


I think it matter whatever the size of the problem. If the matrix isn't full rank you will not fully explore the space and your PCA will be defficient some dimensions.

"Andrew Willis" wrote in message <ivot24$i26$1@newscl01ah.mathworks.com>...
> lg@rndee.dk (Lars Gregersen) wrote in message <3d6fc2c6.11637794@news.cybercity.dk>...
> > On Tue, 27 Aug 2002 07:29:22 -0400, "Gareth Edwards"
> > <gareth.edwards@image-metrics.com> wrote:
> >
> > >John is right of course - I should have said I'm doing PCA on
> > >2000x10000 data matrix. Anyway, it's too big for 'eig' or 'svd' even
> > >though it can be mades a 2000x2000 problem.
> >
> > Many PCA routines assume that the number of rows is larger than the
> > number of columns. For small problems it doesn't matter - for large
> > problems it matters a lot.
> >
> > Fortunately, you can simply perform the PCA on X' and transform the
> > result into the PCA you would have obtained from doing it on X.
> >
> > Lars
> >
> > Lars Gregersen
> > www.rndee.dk
> > lg@rndee.dk
>
>
> I think it matter whatever the size of the problem. If the matrix isn't full rank you will not fully explore the space and your PCA will be defficient some dimensions.

This is a completely irrelevant statement, that you do
not explore the last few dimensions of noise.

The first few eigenvectors generally have structure to
them, indicating behavior that matters in the problem
under study. Those eigenvectors which you do not
fully expose, because the matrix was rank deficient,
are also random noise.

In the case of a truly rank deficient problem, we can
suppose that all the remaining singular values are
exactly zero. In that case, the corresponding singular
vectors all lie in the null space, and are not even unique.
ANY orthogonal basis of that subspace is equally valid.
As such, we have no information about those vectors
except that they span the null space, no do we really
care what they are. They are noise.

John