File Exchange

## mult_comp_perm_t1(d​ata,n_perm,tail,alp​ha_level,mu,reports​,seed_state)

version 1.8 (14.3 KB) by

One sample/paired samples permutation t-test with correction for multiple comparisons

4.83333
6 Ratings

Updated

Permutation test of the null hypothesis that a set of data was sampled from a symmetric distribution with a particular mean. The test is based on a t-statistic and can be applied to situations in which a one sample or paired sample/repeated measures t-test is appropriate. Note, that this test is more general than parametric t-tests in that it does not assume that the data were sampled from a Gaussian distribution.
This function can perform the test on one variable or simultaneously on multiple variables. When applying the test to multiple variables, the "tmax" method is used for adjusting the p-values of each variable for multiple comparisons (Blair & Karniski, 1993). Like Bonferroni correction, this method adjusts p-values in a way that controls the family-wise error rate. However, the permutation method will be more powerful than Bonferroni correction when different variables in the test are correlated.
References:
Blair, R.C. & Karniski, W. (1993) An alternative method for significance testing of waveform difference potentials. Psychophysiology.

Ditrich

zong zhang

peggy

David Groppe

### David Groppe (view profile)

Thanks for the comment, Kielan. I hadn't realized that de2bi.m had been discontinued by Mathworks. I just uploaded a revised version of the function that should work.

Kielan

### Kielan (view profile)

Nice file, but I think there is a bug in the exact test part (for N <= 12) which didn't appear to be testing all possible permutations. Possible fix copied next:

if exact,
%Use all possible permutations

%UPDATED
for i = 1:n_obs
sets{i} = [-1 1];
end

c = cell(1, numel(sets));
[c{:}] = ndgrid( sets{:} );
AllPerms = cell2mat( cellfun(@(v)v(:), c, 'UniformOutput',false) )';

%/UPDATE

mxt=zeros(1,n_perm);
for perm=1:n_perm
if ~rem(perm,100)
if reports,
if ~rem(perm-100,1000)
fprintf('%d',perm);
else
fprintf(', %d',perm);
end
if ~rem(perm,1000)
fprintf('\n');
end
end
end
%set sign of each participant's data

%UPDATED

% if exist('de2bi.m','file') %% ?? check
% temp=de2bi(perm-1);
% else %% check ??
% temp=perm-1; %% check ??
% end %% check ??
% n_temp=length(temp);
% sn=-ones(n_obs,1);
% sn(1:n_temp,1)=2*temp-1;

sn = AllPerms(:,perm);
%/UPDATE

sn_mtrx=repmat(sn,1,n_var);
d_perm=data.*sn_mtrx;

%computes t-score of permuted data across all channels and time points
sm=sum(d_perm,1);
mn=sm/n_obs;
sm_sqrs=sum(d_perm.^2,1)-(sm.^2)/n_obs;
stder=sqrt(sm_sqrs)/sqrt_nXnM1;
t=mn./stder;

%get most extreme t-score
[dummy mxt_id]=max(abs(t));
mxt(perm)=t(mxt_id); %get the most extreme t-value with its sign (+ or -)
end
else

Pierre Mégevand

Joana Paiva

### Joana Paiva (view profile)

Hi,

I would like to know if anyone knows if it possible to perform a repeated measures ANOVA (more than 2 conditions per subject) using your function 'mult_comp_perm_t1' and how I can do that. In the function's description it says that:

Paired-Sample/Repated Measures Example:
% >> dataA=randn(16,5); %data from Condition A (5 variables, 16 observations)
% >> dataA(:,1:2)=dataA(:,1:2)+1; %mean of first two variables is 1
% >> dataB=randn(16,5); %data from Condition B (all variables have mean of 0)
% >> dif=dataA-dataB; %difference between conditions
% >> [pval, t_orig, crit_t, est_alpha, seed_state]=mult_comp_perm_t1(dif,50000);

But it refers to Repeated Measures ANOVA? The example only considers two conditions... for computing more than that, can I taking the differences among them (cond1-cond2-cond3-cond4)?

Thank you,

Cecile

### Cecile (view profile)

Hi,
I have a question. If est_alpha value is 0 (I have 12 observations). the p-value is still correct (in my case p=0)? Should I use and other test ? change a parameter?
Thanks a lot.

Cecile

### Cecile (view profile)

very nice..Thanks

Jiachen Zhuo

### Jiachen Zhuo (view profile)

Nice program. Thank you.

 8 Mar 2016 1.8 Minor change to syntax for using RandStream 8 Mar 2016 1.7 Call to RandStream to get random seed would always use outdated syntax. Now it uses syntax appropriate for MATLAB R2015 and it should be backwards compatible. Let me know if this is not the case. (Thanks for finding the bug Alexa!) 19 Dec 2015 1.6 Comments updated to reference new function for generating multiple comparison adjusted confidence intervals. 25 Oct 2015 1.5 Documentation updated. 25 Oct 2015 1.4 Function documentation updated. 1 Jul 2015 1.3 Command line output cleaned up. 25 Jun 2015 1.3 Function no longer requires de2bi.m, which apparently has been discontinued. Lack of this function would produce errors when # of obs was <=12. Thanks Kielan! 17 Sep 2012 1.2 Comments updated. 3 Feb 2011 1.1 Karniski reference was misspelled as "Karnisky" and has been fixed. 'alpha_level' input option and 'crit_t' and 'est_alpha' output options added.
##### MATLAB Release
MATLAB 7.8 (R2009a)

Play today