I think there are some mistake in this implementation, the last step the feature vector feature dimension reduction procedure is incorrect, since you can not do it in this way. If you do it in this way, how can you tell the difference between PCA and KPCA. we should do it by using inner product form.
Sorry Enrique I don't understand your second point. Surely it doesn't matter whether you normalize just the non-zero eigenvalues or all of the eigenvalues since the non-zero eigenvalues won't change the projections? Have I missed something? Thanks.
Sorry about my last two blank comments. Mouse double-click errors.
Very nice code! However I have a couple of subtle comments:
1) When you center the Kernel matrix be sure to divide the "ones" matrix by the number of samples i.e.
Line 43: one_mat = ones(size(K));
Line 43: one_mat = ones(size(K))./size(data_in,2);
2) Eigenvector normalization implies dividing each of the columns of the eigenvector matrix by the sqrt of its corresponding eigenvalue. Do so by substituting:
Line 61: eigvec(:,col) = eigvec(:,col)./(sqrt(eig_val(col,col)));
Line 61: eigvec(:,col) = eigvec(:,col)./(sqrt(eigval(col,col)));
3) Also, if you must do eigenvalue sorting, be careful to use only the diagonal of the "Lambda" matrix and not the whole matrix. Use:
Line 63: [dummy, index] = sort(diag(eigval),'descend');
Line 63: [dummy, index] = sort(eig_val,'descend');
You can also see "Learning with Kernels" by B. Scholkopf and A. Smola, Section 14.2 (particularly eqs 14.14 and 14.17). Hope this is helpful.