Path: news.mathworks.com!newsfeed-00.mathworks.com!newsfeed2.dallas1.level3.net!news.level3.com!postnews.google.com!1g2000prg.googlegroups.com!not-for-mail
From: Ray Koopman <koopman@sfu.ca>
Newsgroups: sci.stat.math,comp.soft-sys.matlab
Subject: Re: correlation matrix estimation
Date: Tue, 20 May 2008 01:51:24 -0700 (PDT)
Organization: http://groups.google.com
Lines: 67
Message-ID: <7c10fcf5-c0ad-4226-bb24-b32d8a5b61a3@1g2000prg.googlegroups.com>
References: <e963b995-e578-4c43-b848-711ec847fdf8@c65g2000hsa.googlegroups.com>
NNTP-Posting-Host: 70.68.170.218
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Trace: posting.google.com 1211273485 15886 127.0.0.1 (20 May 2008 08:51:25 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Tue, 20 May 2008 08:51:25 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: 1g2000prg.googlegroups.com; posting-host=70.68.170.218; 
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; 
	rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14,gzip(gfe),gzip(gfe)
Xref: news.mathworks.com sci.stat.math:82572 comp.soft-sys.matlab:469419


On May 16, 1:33 pm, msmscarla...@googlemail.com wrote:
> I have written some code which calculates an EWMA estimate for the
> correlation matrix of two time series. Suppose that X and Y are our
> time series vectors of length N (say N daily observations of two
> market interest rates) with X_1 the oldest observation and X_N the
> most recent one.
>
> First the code transforms X and Y by subtracting off the average, i.e.
>
> X --> X - E(X)
> Y --> Y - E(Y)
>
> Then it concatenates X and Y forming a (Nx2) matrix, let's call it Z.
> Then it multiplies the ith row by lambda*(N-i). The covariance matrix
> is calculated by
>
> Cov = (1-lambda) * Z' * Z
>
> Finally, the correlation matrix is computed in the usual way by
>
> Corr = S*Cov*S
>
> where S is a diagonal matrix whose elements are given by 1/
> sqrt(Cov_ii).
>
> My code seems to work fine for small N, but when I try it for very
> large N (several hundreds) I get nonsensical results. Have I missed
> something obvious?
>
> I include my MATLAB code below.
>
> function [CovMat,CorrMat]=EWMACovariance(X,Y,lambda)
> % COVARIANCE Computes covariances and correlations
> n=size(X,1);
> m=mean(X); % Compute the means
> X=X-m(ones(n,1),:); % Subtract the means
> m = mean(Y);
> Y=Y-m(ones(n,1),:);
> lambdaVector = zeros(n,1);
> for i = 1:n
>    lambdaVector(i) = lambda^(n-i);
> end
> X = [X Y];
> lambdaMatrix = diag(lambdaVector);
> X = lambdaMatrix*X;
> CovMat=(1-lambda)*X'*X; % Compute the covariance
> if nargout==2 % Compute the correlation, if requested
>    s=sqrt(diag(CovMat));
>    CorrMat=CovMat./(s*s');
> end

Let Z be an N x 3 matrix in which row i = [x_i, y_i, 1]*sqrt(w_i).
For exponential weighting, take w_i = a^(N-i), with 0 < a < 1.
Let T = Z'Z. Then sum w_i = t_33,
the weighted means are
  m_1 = t_13/t_33  and  m_2 = t_23/t_33,
the weighted variances are
  s_11 = t_11/t_33 - m_1^2  and  s_22 = t_22/t_33 - m_2^2,
the weighted covariance is
  s_12 = t_12/t_33 - m_1*m_2,
and the weighted correlation is
  r_12 = s_12/sqrt(s_11*s_22).

If you want to adjust the variances and covariance for
degrees of freedom, analogous to dividing by n-1 instead of n,
multiply them by n'/(n'-1), where n' = (sum w_i)^2 / sum w_i^2.
Of course, this will not change the correlation.