File Exchange

image thumbnail

Bonferroni-Holm Correction for Multiple Comparisons

version 1.1 (2.87 KB) by

Adjusts a family of p-values via Bonferroni-Holm method to control probability of false rejections.



View License

Bonferroni-Holm (1979) correction for multiple comparisons. This is a sequentially rejective version of the simple Bonferroni correction for multiple comparisons and strongly controls the family-wise error rate at level alpha.
It works as follows:
1) All p-values are sorted in order of smallest to largest. m is the number p-values.
2) If the 1st p-value is greater than or equal to alpha/m, the procedure is stopped and no p-values are significant. Otherwise, go on.
3) The 1st p-value is declared significant and now the second p-value is compared to alpha/(m-1). If the 2nd p-value is greater than or equal to alpha/(m-1), the procedure is stopped and no further p-values are significant. Otherwise, go on.
4) Et cetera.
As stated by Holm (1979) "Except in trivial non-interesting cases the sequentially rejective Bonferroni test has strictly larger probability of rejecting false hypotheses and thus it ought to replace the classical Bonferroni test at all instants where the latter usually is applied."

Holm, S. (1979) A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. 6, 65-70.

Comments and Ratings (2)

Tejas Patel

Thanks for the information, it is in simple way and enough to work


Ben (view profile)

I think there's a bug in the code. if I see correctly the sequential nature of the testing will only be reflected if you set the adjusted p-value to the max across the current adjusted p and *all* preceding ones (as e.g. indicated in the respective wiki entry - the original paper has nothing on adjusted p-values afais).

now,in line 112 your algorithm picks the max of a given p-value and its immediate predecessor. this would be fine if it was done in a sequential fashion (each column updated in turn). but the way it stands the results can differ from adjusting to the max between all p-values up to the current one. you can end up with a vector of adjusted p-values that has decreasing p-values and significant results although the first test was non-significant (try e.g. [p,h]=bonf_holm([.01:.001:.02])). so in the current version non-significant values will be flagged as significant. a remedy would be to use a loop (ugly, I know) like:

for iTest=2:m

also, you might want to add a


to avoid p-values >1




Comments updated

MATLAB Release
MATLAB 7.8 (R2009a)

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video