Remove spikes from a matrix.

Hey all!

I've got a huge matrix of 3587 x 5677056, and I've got to find the average (avg) of this matrix and the standart deviation (sd) for removing the spike values that are along the matrix. So, I will remove all the values of my matrix that are greater than (avg + 3 sd). I've build up the following code:

media_total = NaNmean(matriz_total, 'all');
x = (avg + 3sd)
L = bsxfun(@le,media_total,x);
media_total(L) = NaN;

But I'm getting the following error:

Error using size
Dimension argument must be a positive integer scalar within indexing range.
Error in nanmean (line 43)
count = size(x,dim) - sum(nans,dim);

Does anyone know how to solve it?

And aftewards, how can I effectively remove all the values greater than (avg + 3 sd) and substitute these values for NaN?

Thank's very much for any help!!!

Regards, Paulo Beiral.

Answer 1

[Daniel M]

My version of nanmean only goes up to line 22 in R2019a, so I'm not sure what you have. Perhaps someone has written an overloaded version of that function. What happens when you type this into the command window:

which nanmean

Regardless, this is fairly simple. Other than the fact that a matrix that size of type double would be 162 GB.

xm = nanmean(x,'all');
xs = nanstd(x,0,'all');
x(x > xm+3*xs) = NaN;

[Paulo Eduardo Beiral]

Thank's Daniel, I just removed from matlab path the folder which "which nanmean" showed me, and now nanmean is working fine!