How is the output "m" of multcompare after anovan computed?

20 views (last 30 days)
LK1833 on 16 Jan 2020 at 17:58
Edited: Adam Danz on 16 Jan 2020 at 22:09
Let's say I am performing a three-way ANOVA to reveal the effects of "age" (5 levels), "gender" (2 levels), and "study time" (10 levels) on a test score.
I can perform this three-way ANOVA using MATLAB's anovan function (I am using sums of squares type II, because I am only interested in main effects):
[p, tbl, stats, terms] = anovan(testScore, [age, gender, studyTime], 'varnames', {'age'; 'gender'; 'studyTime'}, 'sstype', 2, 'model', 'linear');
Afterwards, I can use MATLAB's multcompare function to perform post-hoc tests:
[comparison, m] = multcompare(stats, 'display', 'off', 'dimension', 1);
Could someone explain to me in detail, what the first column of the output "m" contains? MATLAB's help says that "m" contains the "estimates and their standard errors" - but how are these estimates actually computed? Would it be correct to say that they are the level means of a given factor when accounting for the influence of the other factors?

Adam Danz on 16 Jan 2020 at 21:59
My answer moved here as a comment
From the documentation,
The first column of m contains the estimated values of the means (or whatever statistics are being compared) for each group, and the second column contains their standard errors.
"how are these estimates actually computed"
Standard error is calculated using the stats output of anova1. The code below demonstrates how to recreate m without using multcompare().
% Compute stats
[p,t,stats] = anova1(MPG,Origin,'off');
% Create [mean, standardError] matrix
stderr = sqrt((stats.s^2)./stats.n);
mm = [stats.means(:), stderr(:)];
% Create the same matrix using multcompare()
[c,m] = multcompare(stats);
% Confirm that they match
isequal(m,mm)
% ~any((m(:)-mm(:)) > 1E-8) % In case there's floating point error
---------------------------------------------------------------------------------------------------------------------
LK1833's response moved here
I understand the computation of the "m" output of multcompare after anova1 (that can also easily re-computed by hand without multcompare like in your code) - but I don't understand "m" of multcompare after applying an n-way anovan. The estimated means for a given factor obtained after anovan are clearly different from the estimated means when performing a one-way anova1 for this factor. Any help understanding this difference would be great :-)
Adam Danz on 16 Jan 2020 at 22:07
@LK1833, I moved my answer to the comment section so that the question does not appear as answered.
Computing the estimates for N-Way ANOVA is much more complex. The good news is that we have access to the code. If you open multcompare() and search for [ case 'anovan' ] you see where those computations start.
open multcompare
% use ctrl+f to open the search box
% search for case 'anovan' (line 216 in r2019b)
The key is to understand variables gmeans and gcov. gmeans defines the output m(:,1). To get m(:,2) you just need to compute sqrt(diag(gcov));
It will probably be helpful if you set a breakpoint at the top of that switch-case, run your code, and then go line by line through that section.