# How to replace messy data by using linear interpolation of nearby messy data

2 views (last 30 days)
Danupon Subanapong on 13 Nov 2018
Commented: KSSV on 13 Nov 2018
Hello!! I would like to ask for some help. I have been working with the data processing for weeks. I am quite new to this. I have been struggling to find the solution to my problem. My problem is that I have to import data from txt file which 90 % of them are alright, but some contains messy data. I will show an example of input text file in the attached file. I would like to clean this data by replacing it by the linear interpolation of the nearby existing values. My messy data is either in form of ********** or 0.
I have been seeking for an answer for a while, but no solution has been found so far.
Note that the actual input file that I work with is in format of .out not .txt but I cannot attach it here.
KSSV on 13 Nov 2018
YOu can replace the messy data with NaN and use fillmissing

KSSV on 13 Nov 2018
Replace all ****** values in the text file with NaN's and follow the below code:
data = importdata('data.txt') ;
data = data.data ;
t = data(:,1) ;
s1 = data(:,2) ;
s2 = data(:,3) ;
%% Remove outliers in s1
stdDev = nanstd(s1) ;% Compute standard deviation
meanValue = nanmean(s1) ; % Compute mean
zFactor = 1.5; % or whatever you want.
% Create a binary map of where outliers live.
outliers = abs(s1-meanValue) > (zFactor * stdDev);
s1(outliers) = NaN ;
% GEt NaN filled
idx = isnan(s1) ;
s1(idx) = interp1(t(~idx),s1(~idx),t(idx)) ;
%% Remove outliers in s2
stdDev = nanstd(s2) ;% Compute standard deviation
meanValue = nanmean(s2) ; % Compute mean
zFactor = 1.5; % or whatever you want.
% Create a binary map of where outliers live.
outliers = abs(s2-meanValue) > (zFactor * stdDev);
s2(outliers) = NaN ;
% GEt NaN filled
idx = isnan(s2) ;
s2(idx) = interp1(t(~idx),s2(~idx),t(idx)) ;
KSSV on 13 Nov 2018
YOu can also have a look on fillmissing.

R2018a

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!