Code covered by the BSD License  

Highlights from
Cohen's kappa

4.5
4.5 | 2 ratings Rate this file 71 Downloads (last 30 days) File Size: 3.62 KB File ID: #15365 Version: 1.3

Cohen's kappa

by

 

20 Jun 2007 (Updated )

Compute the Cohen's kappa

| Watch this File

File Information
Description

Cohen's kappa coefficient is a statistical measure of inter-rater reliability. It is generally thought to be a more robust measure than simple percent agreement calculation since Kappa takes into account the agreement occurring by chance.
Kappa provides a measure of the degree to which two judges, A and B, concur in their respective sortings of N items into k mutually exclusive categories. A 'judge' in this context can be an individual human being, a set of individuals who sort the N items collectively, or some non-human agency, such as a computer program or diagnostic test, that performs a sorting on the basis of specified criteria. The original and simplest version of kappa is the unweighted kappa coefficient introduced by J. Cohen in 1960. When the categories are merely nominal, Cohen's simple unweighted coefficient is the only form of kappa that can meaningfully be used. If the categories are ordinal and if it is the case that category 2 represents more of something than category 1, that category 3 represents more of that same something than category 2, and so on, then it is potentially meaningful to take this into account, weighting each cell of the matrix in accordance with how near it is to the cell in that row that includes the absolutely concordant items. This function can compute a linear weights or a quadratic weights

The output of this function is:
- Observed agreement percentage
- Random agreement percentage
- Agreement percentage due to true concordance
- Residual not random agreement percentage
- Cohen's kappa
- kappa error
- kappa confidence interval
- Maximum possible kappa
- k observed as proportion of maximum possible
- k benchmarks by Landis and Koch
- z test results

You can visit my homepage http://home.tele2.it/cardillo
My profile on XING http://www.xing.com/go/invita/13675097
My profile on LinkedIN http://it.linkedin.com/in/giuseppecardillo

Acknowledgements

This file inspired Cohen's Kappa (With Customizable Weightings).

MATLAB release MATLAB 7.3 (R2006b)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (9)
13 Jul 2015 Giuseppe Cardillo

1 & 2) The confusion matrix is a square matrix so the function will compute the Kappa. The Cohen's kappa is used to test the agreement between judges. If they can classify "objects" into 16 categories you will have a 16x16 square matrix: on the main diagonal you will have "objects" that both judges will classify in the same category.
3) No and read the help section

Comment only
12 Jul 2015 Adrian

Adrian (view profile)

I have a confusion matrix (dimension 16x16) resulted from a classification in 16 classes.

I use >> kappa(cf_mat);

1) If i give this matrix to your function will calculate kappa coefficient for this classification? You only specify X as square data matrix, not as a confusion matrix.

2) Your function works also on multi-class?

3) Do i need to provide weights if the classes are not balanced?

Thank you!

Comment only
12 Jul 2015 Adrian

Adrian (view profile)

I have a confusion matrix (dimension 16x16) resulted from a classification in 16 classes.

1) If i give this function to your function will calculate kappa coefficient for this classification? You only specify X as square data matrix, not as a confusion matrix.

2) Your function works also on multi-class?

Thank you!

07 Jul 2015 Giuseppe Cardillo

And how could help you? The error message is clear

Comment only
07 Jul 2015 Shaveta Arora

Thanks Giuseppe Cardillo
I am getting error
kappa at 98
Warning: all X values must be numeric and finite
pls help

Comment only
01 Jun 2015 Giuseppe Cardillo

because you saved the file into a directory that is unreachable by matlab

Comment only
31 May 2015 Shaveta Arora

this function is giving error:
Undefined function or method 'kappa' for input arguments of type 'double'

Comment only
28 Nov 2008 tzur Karelitz

I think there is an error in the quadratic loop. small caps j needs to be J.

Comment only
04 Jul 2007 Stefano Cavazza  
Updates
25 Sep 2007

Weighted k calculation added

03 Apr 2008

Introduction of asyntotic calculation of variance for large population. Some comment added.

12 Jun 2008

NORMINV was replaced by ERFCINV so Statistics Toolbox is no more needed

12 Nov 2008 1.1

Changes in help section

09 Dec 2008 1.2

correction after tzur Karelitz observation

23 Dec 2009 1.3

Changes in description

Contact us