Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
moving window that ignores NaN

Subject: moving window that ignores NaN

From: Young Ryu

Date: 16 Aug, 2010 18:08:04

Message: 1 of 4

Hi

I have an array that includes NaN:

A=rand(1000, 1000);
A(randsample(size(A, 1), 100), randsample(size(A, 1), 100))=NaN;

I'd like to keep the data array size same, but to use a moving window (10-by-10 pixels) average that ignores NaN. Very inefficient code is like this:

for i=1:size(A, 1)/10
for j=1:size(A, 2)/10
A((i-1)*10+1:i*10, (j-1)*10+1:j*10)=nanmean(nanmean((A((i-1)*10+1:i*10, (j-1)*10+1:j*10))));
end
end

Could you recommend FAST way to do this?
Thank you!

Subject: moving window that ignores NaN

From: Sean

Date: 16 Aug, 2010 18:47:31

Message: 2 of 4

"Young Ryu" <ryuyr77@gmail.com> wrote in message <i4buq4$cni$1@fred.mathworks.com>...
> Hi
>
> I have an array that includes NaN:
>
> A=rand(1000, 1000);
> A(randsample(size(A, 1), 100), randsample(size(A, 1), 100))=NaN;
>
> I'd like to keep the data array size same, but to use a moving window (10-by-10 pixels) average that ignores NaN. Very inefficient code is like this:
>
> for i=1:size(A, 1)/10
> for j=1:size(A, 2)/10
> A((i-1)*10+1:i*10, (j-1)*10+1:j*10)=nanmean(nanmean((A((i-1)*10+1:i*10, (j-1)*10+1:j*10))));
> end
> end
>
> Could you recommend FAST way to do this?
> Thank you!

A few things:
First off for your loop iterator don't recalculate everything every time:
i.e.
for ii = 1:10:size(A,1)
   for jj = 1:10:size(A,1) %note ii and jj since i,j are used as imaginary components
        A(ii:ii+9,jj:jj+9)...
   end
end

Second, is this really what you want? Do you want your new matrix to be 10x10 blocks of all the same number? Or do you want the average value of the window centered at _every_ pixel, not just the ones on a 10x10 grid?

Third:
There may be a faster way to do this by reshaping the whole matrix into a 3d matrix with each page being a 10x10 window and then using the dimensional input to nanmean to calculate the mean. This would bypass both for loops and the second call to nanmean.

A = reshape(A,10,10,[]);
B = nanmean(A,3);

Subject: moving window that ignores NaN

From: Sean

Date: 16 Aug, 2010 19:24:19

Message: 3 of 4

"Sean " <sean.dewolski@nospamplease.umit.maine.edu> wrote in message <i4c143$en3$1@fred.mathworks.com>...
> "Young Ryu" <ryuyr77@gmail.com> wrote in message <i4buq4$cni$1@fred.mathworks.com>...
> > Hi
> >
> > I have an array that includes NaN:
> >
> > A=rand(1000, 1000);
> > A(randsample(size(A, 1), 100), randsample(size(A, 1), 100))=NaN;
> >
> > I'd like to keep the data array size same, but to use a moving window (10-by-10 pixels) average that ignores NaN. Very inefficient code is like this:
> >
> > for i=1:size(A, 1)/10
> > for j=1:size(A, 2)/10
> > A((i-1)*10+1:i*10, (j-1)*10+1:j*10)=nanmean(nanmean((A((i-1)*10+1:i*10, (j-1)*10+1:j*10))));
> > end
> > end
> >
> > Could you recommend FAST way to do this?
> > Thank you!
>
> A few things:
> First off for your loop iterator don't recalculate everything every time:
> i.e.
> for ii = 1:10:size(A,1)
> for jj = 1:10:size(A,1) %note ii and jj since i,j are used as imaginary components
> A(ii:ii+9,jj:jj+9)...
> end
> end
>
> Second, is this really what you want? Do you want your new matrix to be 10x10 blocks of all the same number? Or do you want the average value of the window centered at _every_ pixel, not just the ones on a 10x10 grid?
>
> Third:
> There may be a faster way to do this by reshaping the whole matrix into a 3d matrix with each page being a 10x10 window and then using the dimensional input to nanmean to calculate the mean. This would bypass both for loops and the second call to nanmean.
>
> A = reshape(A,10,10,[]);
This won't work as I expected.
> B = nanmean(A,3);
 %%%This should not have been 3, it should have been 2 calls to 1,2. Though it's still wrong for this reason:

Your method is also not producing the expected results. Your two subsequent calls to nanmean do not return the results you expect. What if one whole column is all nans except for one value and the rest of the columns are real numbers? When you take the nanmean along columns you will be left with a vector where each column has equal weights. This means that the column with only one real value will have equal weight to those with 10 real values. This is wrong. To combat this each 10x10 grid needs to be a vector so that nanmean will return a scalar result taking each real value weighted equally.

Here is an elegant way to do this:
%This is generalized and will work if both dimensions of A are divisible by 10
A = mat2cell(A,repmat(10,1,size(A,1)/10),repmat(10,1,size(A,2)/10));
A = cellfun(@(x)nanmean(x(:)),A);
A = kron(A,ones(10));

Subject: moving window that ignores NaN

From: Young Ryu

Date: 17 Aug, 2010 05:54:05

Message: 4 of 4

"Sean " <sean.dewolski@nospamplease.umit.maine.edu> wrote in message <i4c393$4e5$1@fred.mathworks.com>...
> "Sean " <sean.dewolski@nospamplease.umit.maine.edu> wrote in message <i4c143$en3$1@fred.mathworks.com>...
> > "Young Ryu" <ryuyr77@gmail.com> wrote in message <i4buq4$cni$1@fred.mathworks.com>...
> > > Hi
> > >
> > > I have an array that includes NaN:
> > >
> > > A=rand(1000, 1000);
> > > A(randsample(size(A, 1), 100), randsample(size(A, 1), 100))=NaN;
> > >
> > > I'd like to keep the data array size same, but to use a moving window (10-by-10 pixels) average that ignores NaN. Very inefficient code is like this:
> > >
> > > for i=1:size(A, 1)/10
> > > for j=1:size(A, 2)/10
> > > A((i-1)*10+1:i*10, (j-1)*10+1:j*10)=nanmean(nanmean((A((i-1)*10+1:i*10, (j-1)*10+1:j*10))));
> > > end
> > > end
> > >
> > > Could you recommend FAST way to do this?
> > > Thank you!
> >
> > A few things:
> > First off for your loop iterator don't recalculate everything every time:
> > i.e.
> > for ii = 1:10:size(A,1)
> > for jj = 1:10:size(A,1) %note ii and jj since i,j are used as imaginary components
> > A(ii:ii+9,jj:jj+9)...
> > end
> > end
> >
> > Second, is this really what you want? Do you want your new matrix to be 10x10 blocks of all the same number? Or do you want the average value of the window centered at _every_ pixel, not just the ones on a 10x10 grid?
> >
> > Third:
> > There may be a faster way to do this by reshaping the whole matrix into a 3d matrix with each page being a 10x10 window and then using the dimensional input to nanmean to calculate the mean. This would bypass both for loops and the second call to nanmean.
> >
> > A = reshape(A,10,10,[]);
> This won't work as I expected.
> > B = nanmean(A,3);
> %%%This should not have been 3, it should have been 2 calls to 1,2. Though it's still wrong for this reason:
>
> Your method is also not producing the expected results. Your two subsequent calls to nanmean do not return the results you expect. What if one whole column is all nans except for one value and the rest of the columns are real numbers? When you take the nanmean along columns you will be left with a vector where each column has equal weights. This means that the column with only one real value will have equal weight to those with 10 real values. This is wrong. To combat this each 10x10 grid needs to be a vector so that nanmean will return a scalar result taking each real value weighted equally.
>
> Here is an elegant way to do this:
> %This is generalized and will work if both dimensions of A are divisible by 10
> A = mat2cell(A,repmat(10,1,size(A,1)/10),repmat(10,1,size(A,2)/10));
> A = cellfun(@(x)nanmean(x(:)),A);
> A = kron(A,ones(10));


Dear Sean,

This is a fantastic code!!!! Many thanks.

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us