# Problem having isoutlier detecting anything

23 views (last 30 days)
Vittorio on 9 Aug 2018
Commented: BAPPA MUKHERJEE on 11 Nov 2019
I have what looks like a very easy problem but I cannot seem to solve it. I have a dataset (attached) that has some obvious (to the human eye) outliers.
I cannot get isoutlier to detect it in any way. My attempt is essentially this:
idx = isoutlier(x(:,2),'movmedian',w);
I have put the code in a for loop, spanning all possible values of w, I get at most 3 outliers detected when the window size is 3 and those detected are not actually outliers.
Using movmean instead movmedian detects no outliers for any value of w. I have also played with the threshold factor, but without luck. This seemed to me like a straightforward application for the outlier detection. What am I missing?

Akira Agata on 10 Aug 2018
One possible way to detect this type of outlier would be like this:
% Assuming that data has 2nd order polynomial curve trend
p = polyfit(x(:,1),x(:,2),2);
y = polyval(p,x(:,1));
% Detect outlier in de-trend data
idx = isoutlier(x(:,2)-y);
% Show the result
plot(x(:,1),x(:,2))
hold on
plot(x(idx,1),x(idx,2),'ro')
legend({'Original data','Detected outlier'},'FontSize',14)
BAPPA MUKHERJEE on 11 Nov 2019
Currently I am working on this topic. Can you please help me to load smoothdata function in directory, because in 2012 version its shows undifined function.

Ryan Takatsuka on 9 Aug 2018
The outlier detection generally works best on single data point outliers (not multiple in a row). Your data has a large amount of outliers in a row that have a significant effect on the trend, or moving average, of the data.
In order to detect these outliers, you will need to use a very large moving average window to minimize the effect that the outlier have on it. Additionally, the threshold will need to be modified a bit. I used a window size of w=50 and a threshold of 0.5. This detects the outliers, but also falsely identifies points and the beginning and end of the dataset because the moving average has such a large window.
a = isoutlier(x(:,2), 'movmean', 50, 'ThresholdFactor', 0.5);
It also helps to plot the moving average that is used to calculate the outliers. This is shown in the image:
BAPPA MUKHERJEE on 11 Nov 2019
I am unable to plot the last figure. could you please elaborate this code upto the plotting stage.

BAPPA MUKHERJEE on 11 Nov 2019
Currently I am working on this topic. Can any one help me to load smoothdata function in directory, because in 2012 version its shows undifined function.

R2017a

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!