Eliminate tables with more than 15% NaN data

1 view (last 30 days)
Hey all,
I have C.mat which has 125 tables. I want to remove tables if any of tmax_m or tmin_m or rrr24 or tmin_m columns have more than 15% NaN cells.
C.mat is attached.
Thank you all

Accepted Answer

Adam Danz
Adam Danz on 18 Jan 2020
Edited: Adam Danz on 18 Jan 2020
This cellfun is a bit obnoxious but it produces an output select that is a logical vector the same size as C where true values identify cells that will be eliminated due to having at least 1 column listed in cols that has more than maxNaN percent NaN values.
load('C.mat')
cols ={'tmax_m', 'in_m' 'rrr24' 'tmin_m'};
maxNan = 0.15;
select = cellfun(@(T)any(mean(isnan(T{:,ismember(T.Properties.VariableNames,cols)}),1) > maxNan), C);
C(select) = []; % eliminate selected cells
  1 Comment
BN
BN on 18 Jan 2020
Both codes are accurate, I found this one more understandable for me. Thank you very much, Walter Roberson and Adam Danz

Sign in to comment.

More Answers (1)

Walter Roberson
Walter Roberson on 18 Jan 2020
filetables = load('C.mat');
numtables = numel(filetables);
tablenames = fieldnames(filetables);
droptable = false(1, numtables);
for K = 1 : numtables
thistablename = tablenames{K};
thistable = filetables.(thistablename);
if mean(isnan(thistable.tmax_m)) > 0.15 || ...
mean(isnan(thistable.tmin_m)) > 0.15 || ...
mean(isnan(thistable.rrr24)) > 0.15
droptable(K) = true;
end
end
tables_to_drop = tablenames(droptable);
filetables = rmfield(filetables, tables_to_drop);
save('C_filtered.mat', '-struct', 'filetables');

Categories

Find more on Numeric Types in Help Center and File Exchange

Tags

Products


Release

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!