calculate the averages and standard deviations of massive matrix

7 views (last 30 days)
Genhou
Genhou on 5 Jan 2016
Commented: Walter Roberson on 5 Jan 2016
Hi, everyone. I confront with a problem and hope to get help from you. Now I have a massive matrix, 1000*1000 and I would like to calculate the changes of the averages and standard deviations of the matrix as the dimension goes up. The simple but time-comsuming way is easy. First,I can calculate the matrix of 1000*1000, 500*500,250*250,...... then the averages and standard deviations of each matrix can be calculated. I wonder, however, can anyone help me with a new way to calculate the changes of averages and deviations as the dimension goes up, a method that can calculate with several code, rather than so much repeated work.
  8 Comments
Walter Roberson
Walter Roberson on 5 Jan 2016
Suppose you start with a 1000x1000 matrix and take mean and std over the entire matrix,
m1000 = mean(Array1000(:));
s1000 = std2(Array1000(:));
and then you construct a 500 x 500 matrix of 2 x 2 means:
t500 = conv2(double(Array1000), ones(2,2)/4, 'same');
Array500 = t500(1:2:end, 1:2:end);
and then you take the mean of that:
m500 = mean(Array500(:));
then mathematically this must be the same mean as m1000, to within roundoff error. (The exception would be if you rounded or truncated the fractions when you constructed the 500 x 500 version, then the mean could be slightly different due to loss of precision.)

Sign in to comment.

Answers (1)

Guillaume
Guillaume on 5 Jan 2016
Well, since the smaller matrices are generating by taking the mean of blocks of the larger matrix, the mean of the whole matrices is never going to change. I'm not sure that there is a way to calculate the standard deviation of the small matrices without generating them but if there is, I'm not convinced that it's going to be faster than generating the smaller matrix in the first place.
I'm not sure what your 'simple but time-comsuming way' is, but the following is simple and shouldn't be too time-consuming:
%reduce a matrix size by averaging block of 2x2:
m = rand(1000); %demo data
assert(all(mod(size(m), 2) == 0), 'cannot create 2x2 blocks since matrix has odd size')
smallm = cellfun(@(mm) mean(mm(:)), mat2cell(m, 2 * ones(1, size(m, 1) / 2), 2 * ones(1, size(m, 2) / 2)));
smallstd = std(smallm(:));

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!