View License

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video

Highlights from

4.0 | 1 rating Rate this file 15 Downloads (last 30 days) File Size: 3.99 KB File ID: #25882 Version: 1.5




18 Nov 2009 (Updated )

Friedman test for non parametric two way ANalysis Of VAriance

| Watch this File

File Information

The Friedman test is a non-parametric statistical test developed by the U.S. economist Milton Friedman. Similar to the parametric repeated measures ANOVA, it is used to detect differences in treatments across multiple test attempts. The procedure involves ranking each row (or block) together, then considering the values of ranks by columns. Applicable to complete block designs, it is thus a special case of the Durbin test. The Friedman test is used for two-way repeated measures analysis of variance by ranks. In its use of ranks it is similar to the Kruskal-Wallis one-way analysis of variance by ranks. When the number of blocks or treatments is large (i.e. n > 15 or k > 4), the probability distribution can be approximated by that of a chi-square distribution. If n or k is small, the approximation to chi-square becomes poor and the p-value should be obtained from tables of Q specially prepared for the Friedman test. The MatLab function FRIEDMAN only uses the chi-square approximation. On the contrary, MYFRIEDMAN uses the exact distribution for small size samples; else the F and chi square distributions If the p-value is significant, appropriate post-hoc multiple comparisons test is performed

You can visit my homepage
My profile on XING
My profile on LinkedIN

Required Products Statistics and Machine Learning Toolbox
MATLAB release MATLAB 7.6 (R2008a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (16)
08 Aug 2013 Giuseppe Cardillo

Sorry, but you should ask this to a statistics...I'm a biochemist that wrote a script using the algorithm of Fisher's least significant difference method (Conover WJ, Practical Nonparametric Statistics (3rd edition). Wiley 1999).
You can find it on

Comment only
06 Aug 2013 Pegah Hosseini

Hi, I am a bit confused with the multiple comparison part. If I have 3 treatments and I get a matrix like [0 0 0;1 0 0;0 1 0], then it means that there is a significant difference between treatments (1,2) & (2,3)?
The reason I am this confused is that I apply both friedman of matlab and myfriedman on a set of random 10x3 matrices(say 100 times). Everytime that MATLAB's friedman gives a p-value less than 0.05, myfriedman also reports a significance (they seem to give the same result up to this point). But then the significance difference is different between the multcompare of MATLAB (with Tukey as ctype) and the one in myfriedman. For example, myfriedman gives a difference between (1,2) whereas MATLAB's multcompare gives a difference between (2,3).
I know that the two files use different post-hoc methods but I assumed that one should only be a more conservative form of the other. So, I expected the group differences result of one of these methods to be a subset of the other one not almost completely different group differences.

Comment only
25 Jan 2013 Giuseppe Cardillo

This is becaming a private lesson of statistic... I should claim my name on the paper....
Anyway if you want my advices:
1) write me in private
2) give me ALL informations (what are you really doing? what is your dataset? What are your classifiers?). It is impossible to give some advice on partial informations.

Comment only
25 Jan 2013 Madi

Madi (view profile)

Nemenyi test (alpha == 0.05):

qalpha = [0.000 1.960 2.344 2.569 2.728 2.850 2.948 3.031 3.102 3.164 ...
3.219 3.268 3.313 3.354 3.391 3.426 3.458 3.489 3.517 3.544 ...
3.569 3.593 3.616 3.637 3.658 3.678 3.696 3.714 3.732 3.749 ...
3.765 3.780 3.795 3.810 3.824 3.837 3.850 3.863 3.876 3.888 ...
3.899 3.911 3.922 3.933 3.943 3.954 3.964 3.973 3.983 3.992 ...
4.001 4.009 4.017 4.025 4.032 4.040 4.046];

Then replace cv inside the code with:

cd = qalpha(k)*sqrt(k*(k+1)/(6*b));

So here we get cv using Nemenyi Test.

Please Advice.

Comment only
25 Jan 2013 Madi

Madi (view profile)

Hi, where can I get a real formula for Friedman Test, Critical Value using Fisher's least significant difference method? I need it to put on my report..

Thank You

Comment only
24 Jan 2013 Madi

Madi (view profile)

From this author (Demsar) - the statical comparisons of classifier over the multiple data sets can be used. From his paper we can used:

1) Wilcoxon Signed-Ranks Test
2) Counts of Wins, Losses and Ties: Sign Test
3) Friedman Test

Then he proposed to plot as I give an example.

Please advice. TQ

Comment only
23 Jan 2013 Giuseppe Cardillo

So you can't use any statistical test because they require real number and not probabilities...

Comment only
23 Jan 2013 Madi

Madi (view profile)

I think I can't use Fleiss because my 10classifier will give in percent. For example:
C1=90%, C2=92%, ... , C10=92%, so the total is different for each dataset. From example given, the sum in one row should be same for all row. So, for my case is not..

Comment only
23 Jan 2013 Giuseppe Cardillo

I think it is correct. If you want to test if classifiers are in agreement or not you have to use Fleiss'es test.

Comment only
23 Jan 2013 Madi

Madi (view profile)

Fleiss?.. I need to study more. TQ

I just do some modification just to display the rival method (

x=[115 142 36 91 28; 28 31 7 21 6; 220 311 108 51 117; 82 56 24 46 33; 256 298 124 46 84; 294 322 176 54 86; 98 87 55 84 25];
Treatment = {'1';'2';'3';'4';'5'};
[T,cv] = myfriedman(x);
meansgraph(T, T-cv, T+cv,Treatment);

If wrong please advice... TQ

Comment only
23 Jan 2013 Giuseppe Cardillo

yes you are right. Look well if you need Friedman or Fleiss test.

Comment only
23 Jan 2013 Madi

Madi (view profile)

One more thing, just try to understand the block and treatement:

I have 7 dataset and 10 type of classifier, can I know what are your refering to block and treatement?. Currently I put block as dataset and treartment is my classifier.

Thank You

Comment only
21 Jan 2013 Giuseppe Cardillo

Madi, SPSS seems to use something like the Bonferroni's correction. When the number of comparisons is very high the Bonferroni's correction is too much conservative (is very difficult to have a statistical difference)

Comment only
20 Jan 2013 Madi

Madi (view profile)


Why the result for sum of rank is not divided by 'multiple test attempts'. For example: I have 7 different is true we need to divide by 7? Coz when I test with SPSS, the result is same for the rank only SPSS divide by 7 (for my case).. Please advice

12 Mar 2012 Giuseppe Cardillo

The post-hoc is a nonparametric equivalent to Fisher's least significant difference method (Conover WJ, Practical Nonparametric Statistics (3rd edition). Wiley 1999).
You can find it on

Comment only
10 Mar 2012 Moore betty

Hello, could you please tell me the name of the post-hoc test that you used in code? Bonferroni or Nemenyi test´╝č Many thanks!


Comment only
18 Nov 2009 1.1

Change in help section to cite this file

23 Dec 2009 1.2

Changes in description

17 Mar 2010 1.3

Multiple post-hoc comparisons test added

18 Mar 2010 1.4

bug fixed in input error handling section

23 Apr 2010 1.5

bug correction for small samples computation

Contact us