Skipping time windows in outlier test

I have a large dataset of 1second time-series data in a timetable. In the dataset I also have a binary flag for when there is a filter in place for backgrounding the measurement. I would like to detect outliers for the signal measurement to remove erroneous points before doing any calculations on the data. Is there a simple way to incorporate the filter flag using isoutlier() so that I can ignore those points without the moving median incorporating the last sample time?
The data set has ~10 million points which makes me want to avoid looping through.

5 Comments

Perhaps rmoutliers or isoutlier?
Exactly which times should be removed?
s = load('question_TT.mat')
s = struct with fields:
question_TT: [3000×11 timetable]
tt = s.question_TT;
plot(tt.Time, 'b.-')
grid on;
xlabel('index');
ylabel('time')
I've used isoutlier on variables when "filter_state" == 0 but the timing is discreet. So when I use isoutlier, it's using the MAD from the previous chunk of time when filter_state==0.
You will need to identify the discontinuities and just set the "bad" MAD value at the discontinuities to the next valid "good" value.
So, what I found was that I was not re-synchronizing the outlier flag output from the isoutlier function with the initial flag I used for sample points rather than background points. Mistake by me.
Thanks for your response.

Sign in to comment.

Answers (0)

Categories

Find more on Interpolation in Help Center and File Exchange

Products

Release

R2023a

Asked:

on 31 Oct 2023

Commented:

on 1 Nov 2023

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!