MATLAB Central Newsreader - strange: correlated principal components after orthogonal rotation

strange: correlated principal components after orthogonal rotation

Roland Neumann — Thu, 23 Oct 2008 16:13:02 -0400

I came across a strange behaviour while computing principal components with the Statistics Toolbox (either princomp or pcacov, doesn't matter).

Background is calculating independent software metrics using PCA. This works quite well, when I'm doing like:

EV Lambda =pcacov(corr(M))
...(compute FL out of EV and Lambda)
C= zscore(M)*EV

leads to nicely uncorrelated C (my independent software metrics). Now the strange thing... I want the principal components better suiting the metrics and use orthogonal rotation like varimax or quartimax with:

RotFL = rotatefactors(FL, 'method', 'varimax', 'normalization', 'off')

This is supposed to be (according to help) an orthogonal operation.
But after another:
CRot= zscore(M)*EVRot
I get correlated C.

It's already visible in RotFL: Computing the angles between each 2 column-vektors (A*B/(|A|*|B|) shows no 90? anymore (as for EV and FL).

I'm lost with this supposingly orthogonal rotation and will appreciate any hints. Thank you.

Re: strange: correlated principal components after orthogonal rotation

Roger Stafford — Thu, 23 Oct 2008 17:20:03 -0400

"Roland Neumann" <software-pca@o2online.de> wrote in message <gdq7qd$p1g$1@fred.mathworks.com>...
> I came across a strange behaviour while computing principal components with the Statistics Toolbox (either princomp or pcacov, doesn't matter).
>
> Background is calculating independent software metrics using PCA. This works quite well, when I'm doing like:
>
> EV Lambda =pcacov(corr(M))
> ...(compute FL out of EV and Lambda)
> C= zscore(M)*EV
>
> leads to nicely uncorrelated C (my independent software metrics). Now the strange thing... I want the principal components better suiting the metrics and use orthogonal rotation like varimax or quartimax with:
>
> RotFL = rotatefactors(FL, 'method', 'varimax', 'normalization', 'off')
>
> This is supposed to be (according to help) an orthogonal operation.
> But after another:
> CRot= zscore(M)*EVRot
> I get correlated C.
>
> It's already visible in RotFL: Computing the angles between each 2 column-vektors (A*B/(|A|*|B|) shows no 90? anymore (as for EV and FL).
>
> I'm lost with this supposingly orthogonal rotation and will appreciate any hints. Thank you.
-------
Roland, just because a pair of random variables, x and y, are uncorrelated doesn't mean that a rotation of them will yield uncorrelated variables. Let E{x} = E{y} = E(x*y) = 0 so that x and y are uncorrelated. Then rotate them to u = a*x+b*y and v = -b*x+a*y where a^2+b^2 = 1. Then their (cross) covariance will be:

E{u*v} =
E{(a*x+b*y)*(-b*x+a*y)} =
(a^2-b^2)*E{x*y) + a*b*(E{x^2}-E{y^2}) =
a*b*(E{x^2}-E{y^2})

which will not be zero unless x and y have equal variances. Thus the two quantities u and v are not necessarily uncorrelated.

Roger Stafford

Re: strange: correlated principal components after orthogonal rotation

Roland Neumann — Tue, 04 Nov 2008 09:19:01 -0500

Hello Roger,

Thank you for your fast and precise answer. You helped me alot. When I normalized the Eigenvectors with the Square Root of their Eigenvalues and rotated thereafter, I got uncorrelated Principal components. They all had Variance one.

Now I understand the reason because of your proof.