version 1.2.0.0 (5.3 KB) by
Liber Eleutherios

Compute coefficients of the Weighted Correlation Matrix, as an alternative to CORRCOEFF

WEIGHTEDCORRS returns a symmetric matrix R of weighted correlation coefficients calculated from an input T-by-N matrix Y whose rows are observations and whose columns are variables and an input T-by-1 vector w of weights for the observations. This function may be a valid alternative to CORRCOEF if observations are not all equally relevant and need to be weighted according to some theoretical hypothesis or knowledge.

R = WEIGHTEDCORRS(Y, w) returns a positive semidefinite matrix R, i.e. all its eigenvalues are non-negative (see Example 1 in help section).

WEIGHTEDCORRS is such that WEIGHTEDCORRS(Y, w) == WEIGHTEDCORRS(a * Y + b, w) where a and b are two real numbers (see Example 1 in help section).

Furthermore, the result provided by the function doesn't change if the unit system of each column of Y is changed through an arbitrary affine transformation y = a * x + b, where a and b are two real numbers, with a > 0 (see Example 2 in help section).

If w = ones(size(Y, 1), 1), no difference exists between WEIGHTEDCORRS(Y, w) and CORRCOEF(Y) (see Example 4 in help section).

Reference: F. Pozzi, T. Di Matteo, T. Aste, "Exponential smoothing weighted correlations", The European Physical Journal B, Vol. 85, No 6, 2012. DOI: 10.1140/epjb/e2012-20697-x

***

Example: how to use the function

T = 1000; % number of observations

w = 1:T; % choose weights

Y(:, 1) = (1:T) - T / 2 - 0.5;

Y(:, 2) = rand * Y(:, 1) .^ 2 + rand; % Parabolic relation (symmetric)

r1 = corrcoef(Y) % Traditional Correlation Matrix

r2 = weightedcorrs(Y, w) % Weighted Correlation Matrix

***

Lots of examples in the help section

Liber Eleutherios (2021). Weighted Correlation Matrix (https://www.mathworks.com/matlabcentral/fileexchange/20846-weighted-correlation-matrix), MATLAB Central File Exchange. Retrieved .

Created with
R14

Compatible with any release

**Inspired:**
Weighted Kendall Rank Correlation Matrix, Weighted Covariance Matrix

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!Create scripts with code, output, and formatted text in a single executable document.

Amin MahmoudiHi,

Thank you so much for publishing this function. I need to estimate p values as well, could you please guide me? Thanks

Salma Hassansir , i have 150 images each on has 20 features . can i use this function to find the correlation between all the 20 features for each image is on its own

JosJorrit MLiber EleutheriosHi ASd, thank you for your comment. For confidence intervals and p values try a bootstrap approach. That's what I did in the cited article.

ASdWorks great.

Is it possible to estimate p values as well?

Liber EleutheriosGlad to hear that! I will consider your hints for a new function with more options. Thank you for that - all the best.

Alberto NavarroYes! That was exactly what I was attempting. This solution works great! You should consider making it into a new function and publishing it as an independent file. Maybe instead of an input vector w, the input could be a string with different methods of weight generators. Anyway, thanks a lot again for your submission and your quick help!

Best Regards!

Alberto

Liber EleutheriosDear Alberto,

yes, a vector with entries all equal is interpreted as a vector of uniform weights - in this case weightedcorrs will return exactly the same result as corrcoef. The weights need to be specified - you can use the example provided with the function or the one reported below in this comment, or some other.

A correlation matrix is obtained by using - say - delta_t observations. So, if you have T = 6143 observations and you set delta_t = 250, you will be able to calculate T - delta_t + 1 = 6143 - 250 + 1 = 5894 matrices of moving average correlations.

% Try this example:

% Build time series

T = 6143; % total number of observations

simsp500 = cumsum(randn(T, 1)); % random walk

simcdax = cumsum(randn(T, 1)); % random walk

Y = [simsp500, simcdax]; % matrix of data

delta_t = 250; % running window

% Choose values of weights

decayfactor = -log(0.97); % exponential decay factor

w0 = 1 / sum(exp(((1:delta_t) - delta_t) * decayfactor)); % constant

w = w0 * exp(((1:delta_t) - delta_t) * decayfactor); % weights with exponential decay

% Calculate weighted correlations

HTdummy = zeros(2, 2, T-delta_t+1);

tic

for i=1:(T-delta_t+1)

HTdummy(:,:,i) = weightedcorrs(Y(i:(i + delta_t - 1), :), w);

end

toc

% Elapsed time is 0.117158 seconds.

Is this what you are trying to do?

Alberto NavarroDear Francesco,

I'm sorry. I messed up the input on my question. simsp500 and simcdax are both 6143x1 vectors, hence the input matrix [simsp500 simcdax]. The first time I tried the function, I also created a weights vector (6143x1) w, where all entries were 0.96. I would like to create a time series of the ewma correlation between this two simulated indexes with exponential smoothing of the weights for each of the 6143 periods of time. What I don't understand is if all the entries in the vector w are the same (0.96), will the function adjust and perform the smoothing automatically? Or should I program the vector w to already account for the smoothing by itself? In that case, should I use the exponential decay weights from one of the examples of the function? Is a weight vector where all entries are 0.94 useless? And finally, how could I achieve for the function to deliver a 3D matrix structure [2x2x6143] where every 2x2 matrix is the ewma correlation at each time period?

I hope the main question is clearer now. Thank you very much for your quick response! I really really appreciate the help!

Alberto

Liber EleutheriosDear Alberto,

thank you for your comment.

Please be advised that the function "weightedcorrs" is very similar to Matlab function "corrcoef", with an additional input, the weights of observations.

Note that the first input of the function "weightedcorrs" must be a T-by-N matrix (T observations for each of the N variables) and the second input must be a T-by-1 vector of (non-negative) weights. The weights can have any shape, they do not have to be necessarily exponential.

You might find the help section useful, just type on the Command Window the following line:

help weightedcorrs

As you will see, there are plenty of examples on how to use the function.

Now, from your code I note that you are working with two variables, simsp500 and simcdax, can you clarify what they are?

Apparently, [simsp500(i) simcdax(i)] is a 1-by-2 vector and 0.96 is a scalar - what are you trying to do with these inputs? I don't understand what correlation you hope to obtain between two scalars.

I ignore what a "correlation at the last time period" or "ewma correlation at each time period" are. So, please, can you clarify what you are trying to do?

Alberto NavarroThis delivers only one matrix. I assume its the correlation at the last time period. A correlation matrix for each time period would be much more useful. I attempted to calculate this with the following:

for i=1:6143

HTdummy(:,:,i)=weightedcorrs([simsp500(i) simcdax(i)],0.96)

end

I only get NaN. Could you lend some guidance to use your function to get the ewma correlation at each time period?

Most appreciated!

Tan DemGreat job, but it would be better if the function could create a time series of correlation matrices recursively.