Note: This page has been translated by MathWorks. Please click here

To view all translated materials including this page, select Japan from the country navigator on the bottom of this page.

To view all translated materials including this page, select Japan from the country navigator on the bottom of this page.

Filter outliers using Hampel identifier

The `dsp.HampelFilter`

System
object™ detects
and removes the outliers of the input signal by using the Hampel identifier.
The Hampel identifier is a variation of the three-sigma rule of statistics
that is robust against outliers. For each sample of the input signal,
the object computes the median of a window composed of the current
sample and $$\frac{Len-1}{2}$$ adjacent samples
on each side of current sample. *Len* is the window
length you specify through the `WindowLength`

property.
The object also estimates the standard deviation of each sample about
its window median by using the median absolute deviation. If a sample
differs from the median by more than the threshold multiplied by the
standard deviation, the filter replaces the sample with the median.
For more information, see Algorithms.

The object accepts multichannel inputs, that is, *m*-by-*n* size
inputs, where *m* ≥ 1, and *n* >
1. *m* is the number of samples in each frame (channel),
and *n* is the number of channels. The object also
accepts variable-size inputs. After the object is locked, you can
change the size of each input channel. However, the number of channels
cannot change.

To filter the input signal using a Hampel identifier:

Create a

`dsp.HampelFilter`

object and set the properties of the object.Call

`step`

to filter the signal using the Hampel identifier.

Alternatively, instead of using the `step`

method
to perform the operation defined by the System
object, you can
call the object with arguments, as if it were a function. For example, ```
y
= step(obj,x)
```

and `y = obj(x)`

perform
equivalent operations.

`hampFilt = dsp.HampelFilter`

returns a Hampel
filter object, `hampFilt`

, using the default properties.

`hampFilt = dsp.HampelFilter(Len)`

sets the `WindowLength`

property
to `Len`

.

`hampFilt = dsp.HampelFilter(Len, Lim)`

sets
the `WindowLength`

property to `Len`

and
the `Threshold`

property to `Lim`

.

`hampFilt = dsp.HampelFilter(Name,Value)`

specifies
properties using `Name,Value`

pairs. Unspecified
properties have default values.

**Example**:

hampFilt = dsp.HampelFilter(11,2);

reset | Reset internal states of System object |

step | Remove outliers from input |

Common to All System Objects | |
---|---|

`clone` | Create System object with same property values |

`getNumInputs` | Expected number of inputs to a System object |

`getNumOutputs` | Expected number of outputs of a System object |

`isLocked` | Check locked states of a System object (logical) |

`release` | Allow System object property value changes |

For a given sample of data, *x _{s}*,
the algorithm:

Centers the window of odd length at the current sample.

Computes the local median,

*m*, and standard deviation,_{i}*σ*, over the current window of data._{i}Compares the current sample with

*n*×_{σ}*σ*, where_{i}*n*is the threshold value. If $$\left|{x}_{s}-{m}_{i}\right|>{n}_{\sigma}\times {\sigma}_{i}$$, the filter identifies the current sample,_{σ}*x*, as an outlier and replaces it with the median value,_{s}*m*._{i}

Consider a frame of data that is passed into the Hampel filter.

In this example, the Hampel filter slides a window of length
5 (*Len*) over the data. The filter has a threshold
value of 2 (*n _{σ}*). To
have a complete window at the beginning of the frame, the filter algorithm
prepends the frame with

Current sample:

*x*= 0._{s}Window of data:

*win*= [0 0 0 0 1].Local median:

*m*= median([0 0 0 0 1]) = 0._{i}Median absolute deviation: $$ma{d}_{i}=\mathrm{median}\left(\left|{x}_{i-k}-{m}_{i}\right|,\dots ,\left|{x}_{i+k}-{m}_{i}\right|\right)$$. For this window of data, $$mad=\mathrm{median}\left(\left|0-0\right|,\dots ,\left|1-0\right|\right)=0$$.

Standard deviation:

*σ*=_{i}*κ*×*mad*= 0, where $$\kappa =\frac{1}{\sqrt{2}{\mathrm{erfc}}^{-1}1/2}\approx 1.4826$$._{i}The current sample,

*x*= 0, does not obey the relation for outlier detection._{s}Therefore, the Hampel filter outputs the current input sample,$$\left[\left|{x}_{s}-{m}_{i}\right|=0\right]>\left[\left({n}_{\sigma}\times {\sigma}_{i}\right)=0\right]$$

*x*= 0._{s}

Repeat this procedure for every succeeding sample until the
algorithm centers the window on the $${\left[End-\frac{Len-1}{2}\right]}^{th}$$ sample, marked
as `End`

. Because the window centered on the last $$\frac{Len-1}{2}$$ samples cannot
be full, these samples are processed with the next frame of input
data.

Here is the first output frame the Hampel filter generates:

The seventh sample of the appended input frame, 23, is an outlier. The Hampel filter replaces this sample with the median over the local window [4 9 23 8 12].

[1] Bodenham, Dean. “Adaptive Filtering and Change Detection for Streaming Data.” PH.D. Thesis. Imperial College, London, 2012.

[2] Liu, Hancong, Sirish Shah, and Wei Jiang. “On-line
outlier detection and data cleaning.” *Computers
and Chemical Engineering*. Vol. 28, March 2004, pp. 1635–1647.

[3] Suomela, Jukka. Median Filtering Is Equivalent to Sorting, 2014.

Was this topic helpful?