WEIGHTEDCORRS returns a symmetric matrix R of weighted correlation coefficients calculated from an input T-by-N matrix Y whose rows are observations and whose columns are variables and an input T-by-1 vector w of weights for the observations. This function may be a valid alternative to CORRCOEF if observations are not all equally relevant and need to be weighted according to some theoretical hypothesis or knowledge.
R = WEIGHTEDCORRS(Y, w) returns a positive semidefinite matrix R, i.e. all its eigenvalues are non-negative (see Example 1 in help section).
WEIGHTEDCORRS is such that WEIGHTEDCORRS(Y, w) == WEIGHTEDCORRS(a * Y + b, w) where a and b are two real numbers (see Example 1 in help section).
Furthermore, the result provided by the function doesn't change if the unit system of each column of Y is changed through an arbitrary affine transformation y = a * x + b, where a and b are two real numbers, with a > 0 (see Example 2 in help section).
If w = ones(size(Y, 1), 1), no difference exists between WEIGHTEDCORRS(Y, w) and CORRCOEF(Y) (see Example 4 in help section).
Reference: F. Pozzi, T. Di Matteo, T. Aste, "Exponential smoothing weighted correlations", The European Physical Journal B, Vol. 85, No 6, 2012. DOI: 10.1140/epjb/e2012-20697-x
Example: how to use the function
T = 1000; % number of observations
w = 1:T; % choose weights
Y(:, 1) = (1:T) - T / 2 - 0.5;
Y(:, 2) = rand * Y(:, 1) .^ 2 + rand; % Parabolic relation (symmetric)
r1 = corrcoef(Y) % Traditional Correlation Matrix
r2 = weightedcorrs(Y, w) % Weighted Correlation Matrix
Lots of examples in the help section
sir , i have 150 images each on has 20 features . can i use this function to find the correlation between all the 20 features for each image is on its own
Hi ASd, thank you for your comment. For confidence intervals and p values try a bootstrap approach. That's what I did in the cited article.
Is it possible to estimate p values as well?
Glad to hear that! I will consider your hints for a new function with more options. Thank you for that - all the best.
Yes! That was exactly what I was attempting. This solution works great! You should consider making it into a new function and publishing it as an independent file. Maybe instead of an input vector w, the input could be a string with different methods of weight generators. Anyway, thanks a lot again for your submission and your quick help!
yes, a vector with entries all equal is interpreted as a vector of uniform weights - in this case weightedcorrs will return exactly the same result as corrcoef. The weights need to be specified - you can use the example provided with the function or the one reported below in this comment, or some other.
A correlation matrix is obtained by using - say - delta_t observations. So, if you have T = 6143 observations and you set delta_t = 250, you will be able to calculate T - delta_t + 1 = 6143 - 250 + 1 = 5894 matrices of moving average correlations.
% Try this example:
% Build time series
T = 6143; % total number of observations
simsp500 = cumsum(randn(T, 1)); % random walk
simcdax = cumsum(randn(T, 1)); % random walk
Y = [simsp500, simcdax]; % matrix of data
delta_t = 250; % running window
% Choose values of weights
decayfactor = -log(0.97); % exponential decay factor
w0 = 1 / sum(exp(((1:delta_t) - delta_t) * decayfactor)); % constant
w = w0 * exp(((1:delta_t) - delta_t) * decayfactor); % weights with exponential decay
% Calculate weighted correlations
HTdummy = zeros(2, 2, T-delta_t+1);
HTdummy(:,:,i) = weightedcorrs(Y(i:(i + delta_t - 1), :), w);
% Elapsed time is 0.117158 seconds.
Is this what you are trying to do?
I'm sorry. I messed up the input on my question. simsp500 and simcdax are both 6143x1 vectors, hence the input matrix [simsp500 simcdax]. The first time I tried the function, I also created a weights vector (6143x1) w, where all entries were 0.96. I would like to create a time series of the ewma correlation between this two simulated indexes with exponential smoothing of the weights for each of the 6143 periods of time. What I don't understand is if all the entries in the vector w are the same (0.96), will the function adjust and perform the smoothing automatically? Or should I program the vector w to already account for the smoothing by itself? In that case, should I use the exponential decay weights from one of the examples of the function? Is a weight vector where all entries are 0.94 useless? And finally, how could I achieve for the function to deliver a 3D matrix structure [2x2x6143] where every 2x2 matrix is the ewma correlation at each time period?
I hope the main question is clearer now. Thank you very much for your quick response! I really really appreciate the help!
thank you for your comment.
Please be advised that the function "weightedcorrs" is very similar to Matlab function "corrcoef", with an additional input, the weights of observations.
Note that the first input of the function "weightedcorrs" must be a T-by-N matrix (T observations for each of the N variables) and the second input must be a T-by-1 vector of (non-negative) weights. The weights can have any shape, they do not have to be necessarily exponential.
You might find the help section useful, just type on the Command Window the following line:
As you will see, there are plenty of examples on how to use the function.
Now, from your code I note that you are working with two variables, simsp500 and simcdax, can you clarify what they are?
Apparently, [simsp500(i) simcdax(i)] is a 1-by-2 vector and 0.96 is a scalar - what are you trying to do with these inputs? I don't understand what correlation you hope to obtain between two scalars.
I ignore what a "correlation at the last time period" or "ewma correlation at each time period" are. So, please, can you clarify what you are trying to do?
This delivers only one matrix. I assume its the correlation at the last time period. A correlation matrix for each time period would be much more useful. I attempted to calculate this with the following:
I only get NaN. Could you lend some guidance to use your function to get the ewma correlation at each time period?
Great job, but it would be better if the function could create a time series of correlation matrices recursively.