File Exchange

image thumbnail

hhentschke/measures-of-effect-size-toolbox

version 1.6 (0 Bytes) by

Computes diverse effect size statistics including confidence intervals

28 Downloads

Updated

Editor's Note: This file was selected as MATLAB Central Pick of the Week

The Measures of Effect Size (MES) Toolbox is a set of functions which compute a wide range of effect size statistics. The four main toolbox functions cover common analysis designs, including two-sample-, oneway- and twoway- data sets as well as categorical data (tables). Data may be repeated-measures (within-subjects).
MES for contrasts can be computed. Confidence intervals are generated for the large majority of MES, either via bootstrapping or by analytical computation, in part via noncentral t, Chi square or F distributions.
Effect size statistics are complemented by t/Chi square/F statistics and/or full ANOVA tables, which are also provided as output variables.
The toolbox was developed by Harald Hentschke (University of Tübingen) and Maik Stüttgen (University of Bochum) and is accompanied by a paper (Hentschke and Stüttgen, Eur J Neurosci 34:1887-1894, 2011).
Among the ESM available are:
Hedges’ g
Glass’ delta
requivalent (point-biserial correlation)
common language effect size
Cohen’s U1
Cohen’s U3
receiver-operating characteristic
right/left tail ratio
rank-biserial correlation
standardized mean differences for contrasts
eta squared
partial eta squared
omega squared
partial omega squared
risk difference
risk ratio
odds ratio
phi
sensitivity
specificity
positive predictive value
negative predictive value
binomial effect size display
Cramer’s V

For more details please see the documentation.

Comments and Ratings (25)

Harald Hentschke

Hi everyone, I ported the code to GitHub; it will be automatically updated here from GitHub. If you want to report any issue or have ideas for enhancements, I'd appreciate you doing so on Github, which is ideally suited for discussion and collaboration.
Just a quick response to the latest issue: Rainer, thanks for spotting this. I see your point, but according to Nakagawa and Cuthill (2007), "...note that when a paired designed is used, n 1 = n 2 = n so that the denominator can be written as 8(n–1) – 1..." (p. 599), which is the current implementation. That's about the only reference to bias correction for Hedges' g in the case of dependent data I have found so far. Please let me know (see above) if I'm mistaken or there are other grounds for changing the code.

Hello, I really like your toolbox, would love to see it in R ;-)
Maybe I am mistaken, but it seems as if your bias correction in mes.m is not correct for dependent data. You comment:

% correct for bias due to small n (both dependent and independent
% data, Kline 2004 (p. 102, 106); Nakagawa & Cuthill 2007)
biasFac=(1-(3./(4*n1+4*n2-9)));
es=es.*biasFac;

But this is not the case, the general form is j = 1 - (3/(4 *df - 1)), where df are the degrees of freedom. In case of independent data it is df=(n1+n2-2) [then j=biasFac=(1-(3./(4*n1+4*n2-9)))], but for the dependent data case df=npairs-1, where npairs=n1=n2 (Borenstein, 2009; Kline, 2004, 2013; Rosenthal, 1994, and also Nakagawa & Cuthill, but they only explain it in the text).

It would appreaciate if you could check this.

Harald Hentschke

Thanks for spotting the inconsistency - and no worries: version 1.4, described as the most recent version in code and documentation, is truly the most recent version. Version 1.5 must have resulted from me inadvertently uploading the same code twice on 5 Jan 2015, and there is no way for me to set the Update version on the File Exchange site one tick back (I think). I'll try to rectify this in the next update.

jkr

jkr (view profile)

A welcome toolbox that I look forward to exploring.

An administrative note: whether you choose to install the Toolbox directly (option under Download) or download the zip instead and expand and install it yourself, you get version 1.4. VersionHistory_MESToolbox.pdf covers only through v 1.4. Each code module references v 1.4. The comment below:
"1.5
- mixed within/between analyses are now possible with unequal sample sizes along the between-subjects factor
- minor edits of comments in code and documentation
See VersionHistory_MESToolbox.pdf for details"
Is all that is apparently available describing the difference. It is not clear to me how to get v.1.5.
I doubt this will be a problem for me - just wanted to point it out as I found it confusing.

Yamil Vidal

I got it...
You apply correction for bias due to small sample size.
Thanks a lot.

korkam

korkam (view profile)

Bob Spunt

Harald Hentschke

mes1way is intended for statistics on two or more groups, similar to a oneway ANOVA. If you read the first 20 or so lines of the help you'll know how to arrange the input into mes1way (which is a trifle more complicated than with mes).

Oliver Kumar

Nice toolbox!
Is there a way to yous mes1way to compare two variables A and B. Like it is for mes? Thanks

Harald Hentschke

This would indeed be a worthwile addition (thanks for the hint), but for a lack of time we have to restrict ourselves to maintenance and minor edits of the code, at least for now.

Julie

Julie (view profile)

Great toolbox - have you thought of adding one for heterogeneous variances (e.g. Johnston et al 2004 http://www.ncbi.nlm.nih.gov/pubmed/15058860)?

Harald Hentschke

The example you refer to in the documentation has a line break in an unfortunate place, namely right after the second minus sign in the contrast weights matrix. I presume that upon pasting the example in the command window the code line read (incorrectly)

'cWeight',[1 -1 0; 1 0 - 1]

instead of

'cWeight',[1 -1 0; 1 0 -1]

(note the space after the second minus sign in the incorrect line), which is the likely cause of the error. We'll make sure that in the next version of the documentation code line breaks are more reasonably placed.

Jay Buckey

Ran this example in the manual:
mes1way(com_post,'partialeta2','group',group,'cWeight',[1 -1 0; 1 0 - 1],'isDep',1,'nBoot',10000)

and got this error:
Dimensions of matrices being concatenated are not consistent.

Dominika I

Nolan Conaway

korkam

korkam (view profile)

Harald Hentschke

Hi Pat,

it seems that your data are not sorted in a way mes2way expects them to be, although your example suggests they should be. Anyways, let's assume

tmpData=
[1.4 1 1
1.5 1 2
2 1 3
...
is a N by 3 array containing the dependent variable (whatever it is that you record) in the first column and the numbers coding for the levels of the first and second factor in the 2nd and 3rd column, respectively (in your example, groups and blocks). The following lines

tmpData=sortrows(tmpData,[2 3]);
x=tmpData(:,1);
group=tmpData(:,[2 3]);

should do the trick.

Pat

Pat (view profile)

Hello Harald,

I'm trying this toolbox with a two-way ANOVA, with 3 groups and 40 blocks. I'm not sure how to code the X and g matrices. Right now I have something like:

data Groups Blocks # -->> not in the matrix

1.4 1 1
1.5 1 2
2 1 3
.
.
.
1.5 1 40
.
.
.
and so on for the other 2 groups

But it yielded this error:
??? Error using ==> mes2way at 335
mes2way expects samples from each group to form contiguous blocks. Please sort your data accordingly

Can you help me out,

Thanks

Pat

Zhiguo Ma

Excellent toolbox and document.
It makes effect size and its confidence iterval estimation easy to use and understand.
Thanks very much for your code.

Harald Hentschke

Hello James, no, sorry, it's neither possible nor planned for the near future

James

James (view profile)

Is it possible to use this toolbox for ANOVAs with >2 factors?

Harald Hentschke

Hi Jonathan, yes it does: Hedges' g is one manifestation of Cohen's D, maybe the most popular one. For more details on this, please see the documentation.

Jonathan

does this have Cohen's D, by some other name?? could it be added? at least in Psychology, its a widely used effect size estimate. Thanks!

Wow!
Very impressed with the well written code and manual. Thanks a bunch!

Updates

1.6

Ported code to GitHub (to allow for better collaboration on the code) and set up link from GitHub. Changed variable name 'table' in mes1way and mes2way to 'summaryTable'.

1.5

- mixed within/between analyses are now possible with unequal sample sizes along the between-subjects factor
- minor edits of comments in code and documentation

See VersionHistory_MESToolbox.pdf for details

1.4

- mixed within/between analyses are now possible with unequal sample sizes along the between-subjects factor
- minor edits of comments in code and documentation
See VersionHistory_MESToolbox.pdf for details

1.3

- implementation of exact confidence intervals for g1
- option to plot data in twoway analysis
- enhancement & correction of documentation

See VersionHistory_MESToolbox.pdf for details.

1.2

- added unstandardized MES (mean difference & contrasts)
- enhanced documentation accordingly
- bug fix in mes1way
- streamlined computation of t statistics in mes1way

The changes are explained in more detail in VersionHistory_MESToolbox.pdf

1.1

- some bug fixes
- additions to the documentation
- added example data (which were missing in the previous version, sorry)

Details are listed in VersionHistory.pdf

MATLAB Release
MATLAB 7.5 (R2007b)

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video

Win prizes and improve your MATLAB skills

Play today