How do i remove an outlier?

I have a data set where there is an outlier in the date. I am trying to remove it but the way i am trying is not working. The way i am trying it is by writing that if the next number is so much more higher than the previous it should change it to the previous one. I might not have understood correctly how this works because it is not working. Any help is appreciated!

clear all

close all

clc

load('ACE2-opdracht-2-persoonlijk.mat', 'meetdata');

A = meetdata;

Motortoerental = A (:,1);

Stuurspanning = A (:,2);

Tijd = (0:4000)./50;

figure;

subplot(1,2,1)

plot(Tijd, Motortoerental); hold on

xlabel('Tijd(s)')

ylabel('Toerental(omw/min)')

title('v1');

grid;

subplot(1,2,2)

plot(Tijd, Stuurspanning); hold on

xlabel('Tijd(s)')

ylabel('Stuurspanning(V)')

title('v2');

grid;

Stoorsignaal = 0.02*sin(2*pi*Tijd);

v2_verbeterd = Stuurspanning - Stoorsignaal;

figure;

plot(Tijd,v2_verbeterd);

xlabel('Tijd(s)');

ylabel('Stuurspanning(V)');

grid;

% This is the part where i try to remove the outlier. I include my whole code just to be sure though.

for i = 2:length(Motortoerental)

if ((Motortoerental(i)-Motortoerental(i-1))/0.1)>5

Motortoerental_verbeterd(i) = y(i-1);

end

end

figure;

plot(Tijd,Motortoerental_verbeterd);

xlabel('Tijd(s)')

ylablel('Toerental(omw/min)')

grid;

[Voss]

https://www.mathworks.com/help/matlab/ref/rmoutliers.html

[Wouter]

I must have done something wrong. But when i enter that line of code it doesn't remove the outlier from my plot. Do you have any further suggestions?

[Dyuman Joshi]

What is y here? It has not been defined in the code above

for i = 2:length(Motortoerental)
    if ((Motortoerental(i)-Motortoerental(i-1))/0.1)>5
        Motortoerental_verbeterd(i) = y(i-1);
    end
end

Answer 1

[Constantino Carlos Reyes-Aldasoro]

Use a moving filter, in your case, the best would be to replace each value by the median value of a neighbourhood, that is if you have [1 3 2] the median is 2, if you have [1 1000 2] median is also 2, check this code

a=sin(0:0.1:100);
a(55)=3;
plot(a)

b=movmedian(a,3);
plot(b)

[Dyuman Joshi]

In doing so, the data gets skewed. You can see that the values have changed

a=sin(0:0.1:100);
c=a;
a(55)=3;
b=movmedian(a,3);
b==c

ans = 1×1001 logical array
   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   0   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   1   0   1   1   1   1   1   1   1   1   1   0

nnz(b~=c)

ans = 36

[Wouter]

Thank you kind stranger! It worked!