Code covered by the BSD License  

Highlights from
Bonferroni-Holm Correction for Multiple Comparisons

Be the first to rate this file! 45 Downloads (last 30 days) File Size: 2.87 KB File ID: #28303

Bonferroni-Holm Correction for Multiple Comparisons

by

 

26 Jul 2010 (Updated )

Adjusts a family of p-values via Bonferroni-Holm method to control probability of false rejections.

| Watch this File

File Information
Description

Bonferroni-Holm (1979) correction for multiple comparisons. This is a sequentially rejective version of the simple Bonferroni correction for multiple comparisons and strongly controls the family-wise error rate at level alpha.
 
It works as follows:
1) All p-values are sorted in order of smallest to largest. m is the number p-values.
2) If the 1st p-value is greater than or equal to alpha/m, the procedure is stopped and no p-values are significant. Otherwise, go on.
3) The 1st p-value is declared significant and now the second p-value is compared to alpha/(m-1). If the 2nd p-value is greater than or equal to alpha/(m-1), the procedure is stopped and no further p-values are significant. Otherwise, go on.
4) Et cetera.
 
As stated by Holm (1979) "Except in trivial non-interesting cases the sequentially rejective Bonferroni test has strictly larger probability of rejecting false hypotheses and thus it ought to replace the classical Bonferroni test at all instants where the latter usually is applied."

Reference:
Holm, S. (1979) A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. 6, 65-70.

MATLAB release MATLAB 7.8 (R2009a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (1)
15 Oct 2013 Ben

I think there's a bug in the code. if I see correctly the sequential nature of the testing will only be reflected if you set the adjusted p-value to the max across the current adjusted p and *all* preceding ones (as e.g. indicated in the respective wiki entry - the original paper has nothing on adjusted p-values afais).

now,in line 112 your algorithm picks the max of a given p-value and its immediate predecessor. this would be fine if it was done in a sequential fashion (each column updated in turn). but the way it stands the results can differ from adjusting to the max between all p-values up to the current one. you can end up with a vector of adjusted p-values that has decreasing p-values and significant results although the first test was non-significant (try e.g. [p,h]=bonf_holm([.01:.001:.02])). so in the current version non-significant values will be flagged as significant. a remedy would be to use a loop (ugly, I know) like:

for iTest=2:m
p_adj(iTest)=max(p_adj(1:iTest));
end

also, you might want to add a

p_adj(p_adj>1)=1;

to avoid p-values >1

cheers

Updates
17 Sep 2012

Comments updated

Contact us