Note: This page has been translated by MathWorks. Please click here

To view all translated materials including this page, select Japan from the country navigator on the bottom of this page.

To view all translated materials including this page, select Japan from the country navigator on the bottom of this page.

Filter outliers using Hampel identifier

**Library:**DSP System Toolbox / Filtering / Filter Designs

The Hampel Filter block detects and removes the
outliers of the input signal by using the Hampel identifier. The Hampel
identifier is a variation of the three-sigma rule of statistics that
is robust against outliers. For each sample of the input signal, the
block computes the median of a window composed of the current sample
and $$\frac{Len-1}{2}$$ adjacent samples
on each side of current sample. *Len* is the window
length you specify through the **Window length** parameter.
The block also estimates the standard deviation of each sample about
its window median by using the median absolute deviation. If a sample
differs from the median by more than the threshold multiplied by the
standard deviation, the filter replaces the sample with the median.
For more information, see Algorithms.

For a given sample of data, *x _{s}*,
the algorithm:

Centers the window of odd length at the current sample.

Computes the local median,

*m*, and standard deviation,_{i}*σ*, over the current window of data._{i}Compares the current sample with

*n*×_{σ}*σ*, where_{i}*n*is the threshold value. If $$\left|{x}_{s}-{m}_{i}\right|>{n}_{\sigma}\times {\sigma}_{i}$$, the filter identifies the current sample,_{σ}*x*, as an outlier and replaces it with the median value,_{s}*m*._{i}

Consider a frame of data that is passed into the Hampel filter.

In this example, the Hampel filter slides a window of length
5 (*Len*) over the data. The filter has a threshold
value of 2 (*n _{σ}*). To
have a complete window at the beginning of the frame, the filter algorithm
prepends the frame with

Current sample:

*x*= 0._{s}Window of data:

*win*= [0 0 0 0 1].Local median:

*m*= median([0 0 0 0 1]) = 0._{i}Median absolute deviation: $$ma{d}_{i}=\mathrm{median}\left(\left|{x}_{i-k}-{m}_{i}\right|,\dots ,\left|{x}_{i+k}-{m}_{i}\right|\right)$$. For this window of data, $$mad=\mathrm{median}\left(\left|0-0\right|,\dots ,\left|1-0\right|\right)=0$$.

Standard deviation:

*σ*=_{i}*κ*×*mad*= 0, where $$\kappa =\frac{1}{\sqrt{2}{\mathrm{erfc}}^{-1}1/2}\approx 1.4826$$._{i}The current sample,

*x*= 0, does not obey the relation for outlier detection._{s}Therefore, the Hampel filter outputs the current input sample,$$\left[\left|{x}_{s}-{m}_{i}\right|=0\right]>\left[\left({n}_{\sigma}\times {\sigma}_{i}\right)=0\right]$$

*x*= 0._{s}

Repeat this procedure for every succeeding sample until the
algorithm centers the window on the $${\left[End-\frac{Len-1}{2}\right]}^{th}$$ sample, marked
as `End`

. Because the window centered on the last $$\frac{Len-1}{2}$$ samples cannot
be full, these samples are processed with the next frame of input
data.

Here is the first output frame the Hampel filter generates:

The seventh sample of the appended input frame, 23, is an outlier. The Hampel filter replaces this sample with the median over the local window [4 9 23 8 12].

[1] Bodenham, Dean. “Adaptive Filtering and Change Detection for Streaming Data.” PH.D. Thesis. Imperial College, London, 2012.

[2] Liu, Hancong, Sirish Shah, and Wei Jiang. “On-line
outlier detection and data cleaning.” *Computers
and Chemical Engineering*. Vol. 28, March 2004, pp. 1635–1647.

[3] Suomela, Jukka. Median Filtering Is Equivalent to Sorting, 2014.

Was this topic helpful?