## Documentation Center |

Multiple comparison test

`c = multcompare(stats)c = multcompare(stats,param1,val1,param2,val2,...)[c,m] = multcompare(...)[c,m,h] = multcompare(...)[c,m,h,gnames] = multcompare(...)`

`c = multcompare(stats)` performs a multiple
comparison test using the information in the `stats` structure,
and returns a matrix `c` of pairwise
comparison results. It also displays an interactive graph of the estimates
with comparison intervals around them. See Examples.

In a one-way analysis of variance, you compare the means of
several groups to test the hypothesis that they are all the same,
against the general alternative that they are not all the same. Sometimes
this alternative may be too general. You may need information about
which pairs of means are significantly different, and which are not.
A test that can provide such information is called a *multiple
comparison procedure*.

When you perform a simple *t*-test of one group
mean against another, you specify a significance level that determines
the cutoff value of the *t* statistic. For example,
you can specify the value `alpha` = `0.05` to insure that when there is no
real difference, you will incorrectly find a significant difference
no more than 5% of the time. When there are many group means, there
are also many pairs to compare. If you applied an ordinary *t*-test
in this situation, the `alpha` value would apply
to each comparison, so the chance of incorrectly finding a significant
difference would increase with the number of comparisons. Multiple
comparison procedures are designed to provide an upper bound on the
probability that *any* comparison will be incorrectly
found significant.

The output `c` contains the results of the
test in the form of a six-column matrix. Each row of the matrix represents
one test, and there is one row for each pair of groups. The entries
in the row indicate the means being compared, the estimated difference
in means, a confidence interval for the difference, and a *p*-value
for the hypothesis test that the corresponding mean difference is
different from zero.

For example, suppose one row contains the following entries.

2.0000 5.0000 1.9442 8.2206 14.4971 0.0432

These numbers indicate that the mean of group 2
minus the mean of group 5 is estimated to be 8.2206,
and a 95% confidence interval for the true difference of the means
is [1.9442, 14.4971]. The *p*-value
for the corresponding hypothesis test that the difference of the means
of groups 2 and 5 is significantly different from zero is 0.0432.

In this example the confidence interval does not contain 0.0,
so the difference is significant at the 0.05 level. If the confidence
interval did contain 0.0, the difference would not be significant
at the 0.05 level. The *p*-value of 0.0432 also indicates
that the difference of the means of groups 2 and 5 is significantly
different from zero.

The `multcompare` function also displays a
graph with each group mean represented by a symbol and an interval
around the symbol. Two means are significantly different if their
intervals are disjoint, and are not significantly different if their
intervals overlap. You can use the mouse to select any group, and
the graph will highlight any other groups that are significantly different
from it.

`c = multcompare(stats,param1,val1,param2,val2,...)` specifies
one or more of the parameter name/value pairs described in the following
table.

Parameter | Values |
---|---|

'alpha' | Scalar between |

'display' | Either |

'ctype' | Specifies the type of critical value to use for the multiple
comparison. Values of |

'dimension' | A vector specifying the dimension or dimensions over
which the population marginal means are to be calculated. Use only
if you create |

'estimate' | Specifies the estimate to be compared. The allowable values of estimate depend on the function that was the source of the stats structure, as described in Values of estimate |

`[c,m] = multcompare(...)` returns
an additional matrix `m`. The first
column of `m` contains the estimated
values of the means (or whatever statistics are being compared) for
each group, and the second column contains their standard errors.

`[c,m,h] = multcompare(...)` returns
a handle `h` to the comparison graph. Note that the
title of this graph contains instructions for interacting with the
graph, and the *x*-axis label contains information
about which means are significantly different from the selected mean.
If you plan to use this graph for presentation, you may want to omit
the title and the *x*-axis label. You can remove
them using interactive features of the graph window, or you can use
the following commands.

title('') xlabel('')

`[c,m,h,gnames] = multcompare(...)` returns `gnames`,
a cell array with one row for each group, containing the names of
the groups.

The intervals shown in the graph are computed so that to a very close approximation, two estimates being compared are significantly different if their intervals are disjoint, and are not significantly different if their intervals overlap. (This is exact for multiple comparison of means from anova1, if all means are based on the same sample size.) You can click on any estimate to see which means are significantly different from it.

The following table describes the allowed values for the parameter `ctype`.

Value | Description |
---|---|

'hsd' or 'tukey-kramer' | Use Tukey's honestly significant difference criterion. This is the default, and it is based on the Studentized range distribution. It is optimal for balanced one-way ANOVA and similar procedures with equal sample sizes. It has been proven to be conservative for one-way ANOVA with different sample sizes. According to the unproven Tukey-Kramer conjecture, it is also accurate for problems where the quantities being compared are correlated, as in analysis of covariance with unbalanced covariate values. |

'lsd' | Use Tukey's least significant difference procedure. This
procedure is a simple t-test. It is reasonable if the preliminary
test (say, the one-way ANOVA |

'bonferroni' | Use critical values from the t distribution, after a Bonferroni adjustment to compensate for multiple comparisons. This procedure is conservative, but usually less so than the Scheffé procedure. |

'dunn-sidak' | Use critical values from the t distribution, after an adjustment for multiple comparisons that was proposed by Dunn and proved accurate by Sidák. This procedure is similar to, but less conservative than, the Bonferroni procedure. |

'scheffe' | Use critical values from Scheffé's S procedure, derived from the F distribution. This procedure provides a simultaneous confidence level for comparisons of all linear combinations of the means, and it is conservative for comparisons of simple differences of pairs. |

The allowable values of the parameter `'estimate'` depend
on the function that was the source of the `stats` structure,
according to the following table.

Source | Values |
---|---|

'anova1' | Ignored. Always compare the group means. |

'anova2' | Either |

'anovan' | Ignored. Always compare the population marginal means
as specified by the |

'aoctool' | Either |

'friedman' | Ignored. Always compare average column ranks. |

'kruskalwallis' | Ignored. Always compare average group ranks. |

The dimension parameter is a vector specifying the dimension
or dimensions over which the population marginal means are to be calculated.
For example, if `dim` = `1`, the
estimates that are compared are the means for each value of the first
grouping variable, adjusted by removing effects of the other grouping
variables as if the design were balanced. If `dim` = `[1
3]`, population marginal means are computed for each combination
of the first and third grouping variables, removing effects of the
second grouping variable. If you fit a singular model, some cell means
may not be estimable and any population marginal means that depend
on those cell means will have the value `NaN`.

Population marginal means are described by Milliken and Johnson
(1992) and by Searle, Speed, and Milliken (1980). The idea behind
population marginal means is to remove any effect of an unbalanced
design by fixing the values of the factors specified by `dim`,
and averaging out the effects of other factors as if each factor combination
occurred the same number of times. The definition of population marginal
means does not depend on the number of observations at each factor
combination. For designed experiments where the number of observations
at each factor combination has no meaning, population marginal means
can be easier to interpret than simple means ignoring other factors.
For surveys and other studies where the number of observations at
each combination does have meaning, population marginal means may
be harder to interpret.

[1] Hochberg, Y., and A. C. Tamhane. *Multiple
Comparison Procedures*. Hoboken, NJ: John Wiley & Sons,
1987.

[2] Milliken, G. A., and D. E. Johnson. *Analysis
of Messy Data, Volume 1: Designed Experiments*. Boca Raton,
FL: Chapman & Hall/CRC Press, 1992.

[3] Searle, S. R., F. M. Speed, and G. A.
Milliken. "Population marginal means in the linear model:
an alternative to least-squares means." *American
Statistician*. 1980, pp. 216–221.

`anova1` | `anova2` | `anovan` | `aoctool` | `friedman` | `kruskalwallis`

Was this topic helpful?