Path: news.mathworks.com!newsfeed-00.mathworks.com!newsfeed2.dallas1.level3.net!news.level3.com!postnews.google.com!59g2000hsb.googlegroups.com!not-for-mail
From: vontressms@cs.com
Newsgroups: sci.stat.math,comp.soft-sys.matlab
Subject: Re: correlation matrix estimation
Date: Mon, 19 May 2008 07:50:38 -0700 (PDT)
Organization: http://groups.google.com
Lines: 84
Message-ID: <4d47d7e8-2bc3-4ec4-8aa0-7056ad4478d7@59g2000hsb.googlegroups.com>
References: <e963b995-e578-4c43-b848-711ec847fdf8@c65g2000hsa.googlegroups.com>
NNTP-Posting-Host: 206.197.217.20
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Trace: posting.google.com 1211208639 31646 127.0.0.1 (19 May 2008 14:50:39 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Mon, 19 May 2008 14:50:39 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: 59g2000hsb.googlegroups.com; posting-host=206.197.217.20; 
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; alconnet; 
Xref: news.mathworks.com sci.stat.math:82567 comp.soft-sys.matlab:469266


On May 16, 3:33=A0pm, msmscarla...@googlemail.com wrote:
> I have written some code which calculates an EWMA estimate for the
> correlation matrix of two time series. Suppose that X and Y are our
> time series vectors of length N (say N daily observations of two
> market interest rates) with X_1 the oldest observation and X_N the
> most recent one.
>
> First the code transforms X and Y by subtracting off the average, i.e.
>
> X --> X - E(X)
> Y --> Y - E(Y)
>
> Then it concatenates X and Y forming a (Nx2) matrix, let's call it Z.
> Then it multiplies the ith row by lambda*(N-i). The covariance matrix
> is calculated by
>
> Cov =3D (1-lambda) * Z' * Z
>
> Finally, the correlation matrix is computed in the usual way by
>
> Corr =3D S*Cov*S
>
> where S is a diagonal matrix whose elements are given by 1/
> sqrt(Cov_ii).
>
> My code seems to work fine for small N, but when I try it for very
> large N (several hundreds) I get nonsensical results. Have I missed
> something obvious?
>
> I include my MATLAB code below.
>
> function [CovMat,CorrMat]=3DEWMACovariance(X,Y,lambda)
> % COVARIANCE Computes covariances and correlations
> n=3Dsize(X,1);
> m=3Dmean(X); % Compute the means
> X=3DX-m(ones(n,1),:); % Subtract the means
> m =3D mean(Y);
> Y=3DY-m(ones(n,1),:);
> lambdaVector =3D zeros(n,1);
> for i =3D 1:n
> =A0 =A0lambdaVector(i) =3D lambda^(n-i);
> end
> X =3D [X Y];
> lambdaMatrix =3D diag(lambdaVector);
> X =3D lambdaMatrix*X;
> CovMat=3D(1-lambda)*X'*X; % Compute the covariance
> if nargout=3D=3D2 % Compute the correlation, if requested
> =A0 =A0s=3Dsqrt(diag(CovMat));
> =A0 =A0CorrMat=3DCovMat./(s*s');
> end

here is an R script to do it. you got the last bit wrong.
Mark

# generate some phoney data
require(mvtnorm)
mean <- c(1,2,3,4)
sigma <- matrix( data=3Dc(3,1,1,1,
                        1,3,1,1,
                        1,1,3,1,
                        1,1,1,3),nrow=3D4)
n<-50
cols<-4
some.data<-rmvnorm(n,mean,sigma)

#get average
avg <- rowsum(some.data, rep(1,n))/n
#center data
centered <- some.data - kronecker(rep(1,n),avg)
#get covariance matrix
cov.mat <- t(centered)%*%centered/(n-1)
#test against R function for covariance matrix
test.cov <- cov.mat-cov(some.data)
test.cov

# get diagonals of covariance matrix, take sqrt, and invert
Id<-diag(cols)
diag(Id)<-1/sqrt(diag(cov.mat))

# multiply by Id on both sides.
corr.mat <- Id%*%cov.mat%*%Id]
#test against built in function
test.cor <- corr.mat-cor(some.data)
test.cor