What is the best way to handle missing data?
2 views (last 30 days)
Show older comments
Hello,
So, basically, our data analysis courses request how to handle data properly, and there's one file in particular that contains missing data (ie. showing -99 while other numbers were positive). In order to proceed to effective statistical analysis, we have to take them into account.
Suppose our file is named 'DATA.dat' from now. Here's my method of resolution :
data = load('DATA.dat');
data(data<=0) = NaN;
But here's the problem: not only I can't compute my data properly, but I'm unable from now on to compute the mean, the median or the standard value. So I was wondering if there was another method to treat the file properly without going through the NaNs?
The main solution that was offered is the following :
NegVal = data<=0;
Years = [1:size(data,1)];
for i=size(data,2)
plot(Years(NegVal(:,i)), data(NegVal(:,i),i))
end
But I really don't understand what it's supposed to do and from the looks of it, it's too complicated to use for the following instructions (dressing a bar plot, etc...).
Thanks in advance!
0 Comments
Answers (1)
Jos (10584)
on 28 Nov 2015
You can take a look at the functions nanmean, nanstd and the like (available in the Statistics Toolbox).
You can also through them out, as in:
originalData = [1 2 3 NaN 5 NaN 7]
isOk = ~isnan(originalData)
newData = originalData(isOk)
mean(newData)
nanmean(originalData) % check
2 Comments
Jos (10584)
on 28 Nov 2015
The best method depends on what you want to do. Note that plot will ignore NaNs:
x = [1 2 3 4 5]
y = [2 4 NaN 8 10]
plot(x,y,'bo-')
See Also
Categories
Find more on Line Plots in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!