4.5

4.5 | 15 ratings Rate this file 433 Downloads (last 30 days) File Size: 398 KB File ID: #38300
image thumbnail

PCA and ICA Package

by

 

24 Sep 2012 (Updated )

Implements Principal Component Analysis (PCA) and Independent Component Analysis (ICA).

| Watch this File

File Information
Description

This package contains functions that implement Principal Component Analysis (PCA) and its lesser known cousin, Independent Component Analysis (ICA).

PCA and ICA are implemented as functions in this package, and multiple examples are included to demonstrate their use.

In PCA, multi-dimensional data is projected onto the singular vectors corresponding to a few of its largest singular values. Such an operation effectively decomposes the input single into orthogonal components in the directions of largest variance in the data. As a result, PCA is often used in dimensionality reduction applications, where performing PCA yields a low-dimensional representation of data that can be reversed to closely reconstruct the original data.

In ICA, multi-dimensional data is decomposed into components that are maximally independent in the negentropy sense. ICA differs from PCA in that the low-dimensional signals do not necessarily correspond to the directions of maximum variance; rather, the ICA components have maximal statistical independence. In practice, ICA can often uncover disjoint underlying trends in multi-dimensional data.

Required Products MATLAB
MATLAB release MATLAB 7.8 (R2009a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (29)
25 Nov 2014 Guilherme

Thanks for sharing your code Mr. Moore.

I'm using part of your ICA function to implement my own.
I wasn't get good results using the weight vector decorrelation as you did so I implemented the Gram-Schmidt decorrelation as proposed by Hyvarinen pq 15 (http://mlsp.cs.cmu.edu/courses/fall2012/lectures/ICA_Hyvarinen.pdf).

Here is the code I inserted right after the weight normalization, still inside the for loop:

% Gram-Schmidt
summ = zeros(nOb,1);
for j = 1 : p-1
wj = W(j,:)';
summ = summ + W(p,:)*wj*wj;
end
W(p,:) = W(p,:) - summ';
W(p,:) = W(p,:)/norm(W(p,:));

25 Sep 2014 Body

I get it. Thanks a lot!

25 Sep 2014 Brian Moore

@Susanne: The for-loop containing "gp" is implementing steps 2-3 of the algorithm on pg. 14. So, in particular, gp is related to g'(.) from the document

24 Sep 2014 Susanne

Ok I think I got a better grasp on it thank you so much your code helped me a lot. I got on question though, what is the 'gp' you're using in your ICA code?

23 Sep 2014 Brian Moore

@Body: No, the "err" vector keeps track of the how much the weight vectors are changing at each iteration (used only for stopping criterion). If you do as you suggested, you'll just be measuring how close each weight vector is to being unit norm (*spoiler alert* - they're always unit norm)

23 Sep 2014 Brian Moore

@Susanne: I suggest you read the help info and comments in the myICA.m function. If you need more detail, you'll need to refer to the literature. For example I recommend

http://mlsp.cs.cmu.edu/courses/fall2012/lectures/ICA_Hyvarinen.pdf

FYI: I'm using the 4-step algorithm on pg. 14 along with the symmetric decorrelation step involving the W matrix from (45) on pg. 15

23 Sep 2014 Body

Thanks for your code! ICA is so difficult to understand. U help a lot! I have a question. Can
err(i) = 1 - w(i,:) * w(i,:)'
instead of
err(i) = 1 - w(i,:) * w_old(i,:)'?

23 Sep 2014 Susanne

I have some trouble to completely understand the ICA algorithmen zou are using. Would you mind heling me out?

17 Sep 2014 Susanne  
16 Sep 2014 Brian Moore

@LionelB: Good point - I'm disappointed in myself for not mentioning bsxfun() - its MATLAB's secret sauce for vectorizing

16 Sep 2014 LionelB

Or even

z0 = bsxfun(@minus,z,sum(z,2)/n);
R = (z0*z0')/(n-1);

:) (Matlab uses this idiom internally, see e.g. the 'cov' function).

Anyhow, I only raised this because I almost gave up on your fine package until I realised the fix was trivial.

16 Sep 2014 Brian Moore

@LionelB: Yes, I'm aware. I wrote the MATLAB code so it could be directly translated into, e.g., C++. If I were maximizing MATLAB efficiency, I would have written

z0 = (1 / (n - 1)) * (z - repmat(mean_z,[1 n]));
R = z0 * z0';

instead of

z0 = z - mean_z*ones(1,n);
R = (z0*z0') / (n-1);

16 Sep 2014 LionelB

Nice package, but the 'myWhiten' routine is a terrible computational bottleneck for large datasets, because of the inefficient sample covariance calculation. This can easily be done without looping: simply de-mean z, e.g.:

z0 = z-mean_z*ones(1,n);

then calculate

R = (z0*z0')/(n-1)

Delivers an orders-of-magnitude speedup.

30 Jul 2014 Ilia

yes i was talking about z_LD, i was afraid that it could be a too strong restriction for the solution, because I'm not really expert with this kind of analysis, and it is the first time i find the possibility to compute the low dimension z.
Thanks you for the answer.

29 Jul 2014 Brian Moore

@Ilia: Are you referring to "z_LD" as described in the myICA help? It's a matrix of the same size as the input "z" that approximates z from a linear combination of the "NUM" independent components in output "z_ic"

29 Jul 2014 Ilia

thank you for sharing the great job,I've a question.
I'm using your 'myICA.m' and i would like to understand what ICA_LD really do.
can you suggest some lecture about it or explain in few words,if it is possible.
thanks you very much

04 Jun 2014 Brian Moore

@Prarinya Check out the FastICA algorithm from

http://mlsp.cs.cmu.edu/courses/fall2012/lectures/ICA_Hyvarinen.pdf

I'm using the 4-step algorithm on pg. 14 along with the symmetric decorrelation step involving the W matrix from (45) on pg. 15

Cheers!

04 Jun 2014 Prarinya Siritanawan

Great contribution. I have a request

Could you please tell us the reference of the algorithm you use? I'm curious about the update rule for creating the transformation matrix 'A' in the code.

I tried to study the ICA algorithm from the ground. However update rule you use are quite different from the document I have. Therefore, I realized that there are more than one approach to update the transformation matrix.

22 Apr 2014 Arjuna

better than FastIca ;)

27 Feb 2014 Bosi  
09 Jan 2014 Abdelrahman

super good

14 Nov 2013 Brian Moore

Pierre,

You must be using an old version of MATLAB (<= 2009b I believe) where ~ is not supported. Please replace the line with something like

[U,S,temp] = svd(w,'econ');

The variable temp is not used; the name is arbitrary

14 Nov 2013 Pierre-Pascal

Hi thanks for providing this code! super useful.

line 97 on my ICA returns an error
[U,S,~] = svd(w,'econ');
it seems to hate the ~ as an output

did I miss something?

30 Aug 2013 Morteza Abdollahzadeh  
12 Jul 2013 Shivakumar

Sir, there is an error at line 31. Can you please solve that? thank you for providing this file.

01 Jul 2013 Jim  
07 Nov 2012 Wang  
15 Oct 2012 Brian Moore

Why the 2 star ranking Eugene? Please provide feedback so I can improve this package!

15 Oct 2012 Eugene  
Updates
09 Oct 2012

Fixing bug in myMultiGaussian(). Needed to use lower triangular Cholesky factorization, not the upper triangular version.

05 Nov 2012

Updating myPCA() documentation

Contact us