Eliminate tables with more than 15% NaN data
1 view (last 30 days)
Show older comments
Hey all,
I have C.mat which has 125 tables. I want to remove tables if any of tmax_m or tmin_m or rrr24 or tmin_m columns have more than 15% NaN cells.
C.mat is attached.
Thank you all
0 Comments
Accepted Answer
Adam Danz
on 18 Jan 2020
Edited: Adam Danz
on 18 Jan 2020
This cellfun is a bit obnoxious but it produces an output select that is a logical vector the same size as C where true values identify cells that will be eliminated due to having at least 1 column listed in cols that has more than maxNaN percent NaN values.
load('C.mat')
cols ={'tmax_m', 'in_m' 'rrr24' 'tmin_m'};
maxNan = 0.15;
select = cellfun(@(T)any(mean(isnan(T{:,ismember(T.Properties.VariableNames,cols)}),1) > maxNan), C);
C(select) = []; % eliminate selected cells
More Answers (1)
Walter Roberson
on 18 Jan 2020
filetables = load('C.mat');
numtables = numel(filetables);
tablenames = fieldnames(filetables);
droptable = false(1, numtables);
for K = 1 : numtables
thistablename = tablenames{K};
thistable = filetables.(thistablename);
if mean(isnan(thistable.tmax_m)) > 0.15 || ...
mean(isnan(thistable.tmin_m)) > 0.15 || ...
mean(isnan(thistable.rrr24)) > 0.15
droptable(K) = true;
end
end
tables_to_drop = tablenames(droptable);
filetables = rmfield(filetables, tables_to_drop);
save('C_filtered.mat', '-struct', 'filetables');
0 Comments
See Also
Categories
Find more on Numeric Types in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!