Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Replace NaNs with previous values

Asked by Johannes on 9 Oct 2012
Latest activity Commented on by Mohammad Sayeed on 14 Jan 2014

Hello,

I have the following problem. I like to replace NaNs with the previous values.

A =

     4     5     6     7     8
    32   NaN   NaN    21   NaN
    12   NaN    12   NaN   NaN
    34   NaN   NaN   NaN   NaN

B =

     4     5     6     7     8
    32     5     6    21     8
    12     5    12    21     8
    34     5    12    21     8

I sloved it like this:

for i = 2:5
[r,c] = find(isnan(A(:,i)));
while sum(isnan(A(:,i)))>0
A(r,i) = A(r-1,i);
end
end

I'm sure there is a way avoiding the for and the while statement. I search for an "elegant" solution.

Someone's able to help me?

2 Comments

Matt Fig on 9 Oct 2012

What if a whole column is nan? Which value will fill it?

Johannes on 9 Oct 2012

If the first value is NaN, everything should be NaN untill a different value appears in the column.

Thanks, Johannes

Johannes

Products

3 Answers

Answer by Matt Fig on 9 Oct 2012
Edited by Matt Fig on 9 Oct 2012

Johannes, notice that your solution will fail if the first value in a column is nan. Rather than looking for a vectorized solution that may end up being rather convoluted (and being slower!), I would simply write a good FOR loop function that can handle all cases. For example, the following solution does not use the FIND function, and only uses simple loops and thus should be very fast:

function A = fill_nans(A)
% Replaces the nans in each column with 
% previous non-nan values.
for ii = 1:size(A,2)
    I = A(1,ii);
    for jj = 2:size(A,1)
        if isnan(A(jj,ii))
            A(jj,ii) = I;
        else
            I  = A(jj,ii);
        end
    end
end

2 Comments

owr on 9 Oct 2012

This is really nice, readable and makes sense. I especially like the fact that you were able to implement it as an in-place function. In a couple quick tests, a "find" based solution doesnt seem to be any worse performance wise, but I still think I like this better because it is really clean. I may use it for myself, thanks for sharing!

Mohammad Sayeed on 14 Jan 2014

Hi I tried to apply your codes but it showed following error: Error using fill_nans (line 4) Not enough input arguments. Can you please tell me how can I correct it?

Matt Fig
Answer by Wayne King on 9 Oct 2012
Edited by Wayne King on 9 Oct 2012

How about:

 A = [ 4     5     6     7     8
    32   NaN   NaN    21   NaN
    12   NaN    12   NaN   NaN
    34   NaN   NaN   NaN   NaN];
 indices = isnan(A);
 A(indices) = 0;
 B = repmat([4 5 6 7 8],size(A,1),1);
 A = A+B.*indices;

1 Comment

Matt Fig on 9 Oct 2012

Johannes comments:

"Solution there:

A =

     4     5     6     7     8
    32     5     6    21     8
    12     5    12     7     8
    34     5     6     7     8

Not good, would need the following: 4 5 6 7 8 32 5 6 21 8 12 5 12 21 8 34 5 12 21 8

Still thanks for you help!"

Wayne King
Answer by owr on 9 Oct 2012

I do this all the time, my code uses for loops, but I dont see anything wrong with for loops. Im sure there are more elegent solutions but this does the trick for me and is more than fast enough:

function datai = backfillnans(data)
% Dimensions
[numRow,numCol] = size(data);
% First, datai is copy of data
datai = data;
% For each column
for c = 1:numCol
    % Find first non-NaN row
    indxFirst = find(~isnan(data(:,c)),1,'first');
    % Find all NaN rows
    indxNaN = find(isnan(data(:,c)));
    % Find NaN rows beyond first non-NaN
    indx = indxNaN(indxNaN > indxFirst);
    % For each of these, copy previous value
    for r = (indx(:))'
        datai(r,c) = datai(r-1,c);
    end
end

2 Comments

Matt Fig on 9 Oct 2012

This seems to fail when a whole column of data is nan.

A = [25   NaN    54    99    20
     3   NaN    92    74    89
     7   NaN   NaN   NaN    82
     75   NaN    43    65    77
     NaN   NaN    15   NaN    38]
owr on 9 Oct 2012

Ah, good catch Matt, thanks for that. Ive been using this for almost 2 years multiple times a day and thats never come up - I guess I never have a full column of nans. It can be fixed I guess by putting an:

if( ~isempty(indxFirst) )

after the line that calculates "indxFirst". Part of me would actually like the whole process to fail so I can figure out why I passed a full column of nans in the first place - that would be symptomatic of a much bigger issue...

Anyways, thanks for taking the time to run and test the code.

owr

Contact us