This example explores different outlier removal filters and uses an electrocardiogram (ECG) signal as input.
There are many different outlier removal techniques because a rigid definition of an outlier does not exist.
The three techniques explored in this example are:
The ECG signal used in this example is taken from the MIT-BIH Arrhythmia Database. The signal is sampled at 360 Hz. The signal was shifted and scaled to convert it from the raw 12-bit ADC values to real-world values.
For more information on ECG signals, please see the example Real-Time ECG QRS Detection.
First, create a stream from the ECG signal using the dsp.MatFileReader. Next, create a scope to visualize the raw and filtered signals.
Fs = 360; frameSize = 500; fileName = 'ecgsig.mat'; winLen = 13; % Window length for the filters. fileReader = dsp.MatFileReader('Filename',fileName, ... 'VariableName','ecgsig','SamplesPerFrame',frameSize); scope = dsp.TimeScope('SampleRate',Fs,'TimeSpanOverrunAction','Scroll', ... 'TimeSpan',2,'YLimits',[-1.5 1.5],'ShowGrid',true, ... 'NumInputPorts',2,'LayoutDimensions',[2 1]); scope.ActiveDisplay = 1; scope.Title = 'Raw Signal'; scope.ActiveDisplay = 2; scope.Title = 'Filtered Signal';
The moving average filter calculates a running mean on the specified window length. This is a relatively simple calculation compared to the other two filters. However, this will smooth both the signal and the outliers. This causes the peak in the ECG signal to be smoothed to roughly a third of its original magnitude.
movAvg = dsp.MovingAverage(winLen); while ~isDone(fileReader) x = fileReader(); y = movAvg(x); scope(x,y); end % Clean up release(scope); reset(fileReader); reset(scope);
The Median Filter is sometimes a better choice since it is less sensitive to outliers than the Moving Average Filter. However, as can be seen in the scope below, it can cause "steps" to appear at extremes in the signal where the local median does not change. This means that the window length of the filter must be carefully considered.
medFilt = dsp.MedianFilter(winLen); while ~isDone(fileReader) x = fileReader(); y = medFilt(x); scope(x,y); end % Clean up release(scope); reset(fileReader); reset(scope);
The Hampel Filter has an additional threshold parameter that can be set. Below, it is set to one, meaning that any sample that is more than one standard deviation away from the local median will be classified as an outlier. Both the threshold and the window length can be changed to remove outliers from the input signal without distorting the original signal.
thres = 1; hampFilt = dsp.HampelFilter(winLen,thres); while ~isDone(fileReader) x = fileReader(); y = hampFilt(x); scope(x,y); end % Clean up release(scope); reset(fileReader); reset(scope);
All three of the above filters can be used for outlier removal. The noise distribution of the outliers and the window length both effect the filter's performance. This must be taken into consideration when selecting a filter for outlier removal in a specific application.
Goldberger AL, Amaral LAN, Glass L, Hausdorff JM, Ivanov PCh, Mark RG, Mietus JE, Moody GB, Peng C-K, Stanley HE. "PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals." Circulation 101(23):e215-e220, 2000.
Moody GB, Mark RG. "The impact of the MIT-BIH Arrhythmia Database." IEEE Eng in Med and Biol 20(3):45-50 (May-June 2001).