Clamp (winsorize) extremal values
This functionality does not run in MATLAB.
[x1, x2, …],
[[x11, x12, …], [x21, x22, …], …],
…], α) returns a copy of [x1, x2,
…] in which all entries smaller than the α quantile
are replaced by this value and likewise for all entries larger than
the 1 - α quantile.
α, i) and
x12,...], [x21, x22,...],...]),
α, i) works on the i-th
entries of the input rows.
Measurement data often contains "outliers," sample points rather far outside the range containing the majority of the points. While expected both from theory and experience, these outliers, for small or medium-sized samples, tend to distort statistical data such as the mean value.
One of the standard methods dealing with this problem for (real)
continuous scales is clamping the outliers.
all data points below or above a given quantile to these quantiles.
This operation is named after its inventor, Charles P. Winsor.
Create a normally distributed sample, slightly contaminated:
r := stats::normalRandom(0, 1, Seed=2): data := [r() $ i = 1..300, 100*r() $ i = 1..2]:
The two extra points distort the data significantly:
stats::winsorize removes this
noise and the image shows more detail:
plot(plot::Histogram2d(stats::winsorize(data, 1/100), Cells=20))
stats::winsorize reduces the standard
deviation of the sample. Keeping in mind that the standard deviation
of the random number generator is 1,
compute that of the data in its various forms:
stats::stdev(data), stats::stdev(stats::winsorize(data, 1/20))
Statistical data: arithmetical expressions. The data to filter on must be real-valued.
Sample of type
Cutoff parameter: a real-valued expression .
Column index: positive integer. The nested list or the sample is winsorized on its i-th column.
The input data with outliers being replaced by the values of quantiles.