subspacea(F,G,A) Finds all min(size(orth(F),2),size(orth(G),2)) principal angles between two subspaces spanned by the columns of matrices F and G in the A-based scalar product x'*A*y, where A is Hermitian and positive definite. COS of principal angles is called canonical correlations in statistics.
[theta,U,V] = subspacea(F,G,A) also computes left and right
principal (canonical) vectors - columns of U and V, respectively.
If F and G are vectors of unit length and A=I, the angle is ACOS(F'*G) in exact arithmetic. If A is not provided as a third argument, than A=I and the function gives the same largest angle as SUBSPACE.m by Andrew Knyazev. MATLAB's SUBSPACE.m function is still badly designed and fails to compute some angles accurately.
The algorithm is described in A. V. Knyazev and M. E. Argentati, Principal Angles between Subspaces in an A-Based Scalar Product: Algorithms and Perturbation Estimates. SIAM J. Scientific Computing, 23 (2002), no. 6, 2009-2041. http://dx.doi.org/10.1137/S1064827500377332