This function can calculate any of the 6 different ICCs defined by McGraw as well as their confidence intervals. In addition a hypothesis test is performed with the null hypothesis that ICC = r0.
[r, LB, UB, F, df1, df2, p] = ICC(M, type, alpha, r0)
M is matrix of observations. Each row is an object of measurement and each column is a judge or measurement.
'type' is a string that can be one of the six possible codes for the desired type of ICC:
'1-1': The degree of absolute agreement among measurements made on randomly seleted objects. It estimates the correlation of any two measurements.
'1-k': The degree of absolute agreement of measurements that are averages of k independent measurements on randomly selected objects.
'C-1': case 2: The degree of consistency among measurements. Also known as norm-referenced reliability and as Winer's adjustment for anchor points. case 3: The degree of consistency among measurements maded under the fixed levels of the column factor. This ICC estimates the corrlation of any two measurements, but when interaction is present, it underestimates reliability.
'C-k': case 2: The degree of consistency for measurements that are averages of k independent measurements on randomly selected onbjectgs. Known as Cronbach's alpha in psychometrics. case 3: The degree of consistency for averages of k independent measures made under the fixed levels of column factor.
'A-1': case 2: The degree of absolute agreement among measurements. Also known as criterion-referenced reliability. case 3: The absolute agreement of measurements made under the fixed levels of the column factor.
'A-k': case 2: The degree of absolute agreement for measurements that are averages of k independent measurements on randomly selected objects. case 3: he degree of absolute agreement for measurements that are based on k independent measurements made under the fixed levels of the column factor.
ICC is the estimated intraclass correlation. LB and UB are upper and lower bounds of the ICC with alpha level of significance.
In addition to estimation of ICC, a hypothesis test is performed with the null hypothesis that ICC = r0. The F value, degrees of freedom and the corresponding p-value of the this test are reported.
(The code names for ICC classes correspond to the Table 7 in the reference).
Reference: McGraw, K. O., Wong, S. P., "Forming Inferences About Some Intraclass Correlation Coefficients", Psychological Methods, Vol. 1, No. 1, pp. 30-46, 1996
Is it also possible to determine the ICC, when there are multiple NaN values in my matrix?
When trying to run the function, I get:
[r, LB, UB, F, df1, df2, p]=ICC([datalr datarl], 'A-1', 0.05, 0)
Undefined function or variable 'lgamma'.
Error in betapdf (line 60)
+ lgamma (a + b) - lgamma (a) - lgamma (b));
Error in betainv (line 87)
h = (betacdf (y_old, a, b) - x) ./ betapdf (y_old, a, b);
Error in finv (line 58)
inv(k) = ((1 ./ betainv (1 - x(k), n/2, m/2) - 1) * n / m);
Error in ICC>ICC_case_C_k (line 137)
FL = (MSR/MSE) / finv(1-alpha/2, n-1, (n-1)*(k-1));
Error in ICC (line 75)
[r, LB, UB, F, df1, df2, p] = ICC_case_C_k(MSR, MSE, MSC, MSW, alpha, r0, n, k);
It seems like there is a problem in the betapdf function, the function lgamma does not exist. Any tips on how to solve this? I am running matlab 18.104.22.1683654 on Mac.
Can someone please help me understand how to arrange my matrix for this code? Can we have trial repetitions (same number of reps) in the rows?
For example, I have data based on 3 people and I want to check the consistency between 5 types of anatomical measurements taken from them. I took their measurements 15 times. Can I use type 'C-k', where the matrix M has 45 rows (each person's data concatenated on top of the other) and 5 columns (each column is a different anatomical measurement type)?
@lee ni and @Louise Nielsen. If you use the app "The Unarchiver" it should be no problem. Or you can just click the "Functions"-tab above and copy the function into a new file on your computer.
I have the same issue with Louise Nielsen that I cannot unzip the file on my mac. Any clues?
Can you comment on Victor's suggestions?
Hi, the confidence intervals in ICC(2,k) aren't accurate. The original article had a mistake that later the original authors corrected. See "Forming inferences about some intraclass correlations coefficients": Correction." (DOI: 10.1037/1082-989X.1.4.390)
This implies that lines lines 175 and 176 should be changed to:
a = r/(n*(1-r));
b = 1+r*(n-1)/(n*(1-r));
and df2 in line 180 should be computed as "df2 = (n - 1) * (k - 1);" and not as "df2 = v"
Also, can I translate the code to python and upload to scipy project (www.scipy.org) retaining the original license note?
I would like to use this function but for some reason I cannot unzip it. My computer gives the following error message when trying: Error 1 - action is not allowed. I have only had this problem with this zip-file.
Thanks for fixing it!
@Simon The code names or the 'type' argument correspond to the case numbers according to Table 7 in McGraw 1996.
@Erlend: You are right! I fixed the issue. It was due to a bug in another function (anova_rm). I removed the dependency and tested the function by comparing the results with IRR package in R.
Hi, the code below produces r-values that are not between LB and UB. Have I misunderstood something on the usage of this function?
x = (0:0.1:10)';
y = 0.8*x + rand(size(x));
[r, LB, UB, F, df1, df2, p] =icc([x,y],'1-1',0.05,0.5);
[LB, r, UB]
0.8141 0.9549 0.9108
Thank you very much for sharing this code, it's very useful! However, could you give us the correspondence between the 'type' arguments ('1-1', '1-k', 'C-1', 'C-k', 'A-1', 'A-k') in your function and the cases (Case 1, Case 2, Case 2A, Case 3, Case 3A) from the article you used to code this function?
Thank you very much in advance!
@Matthieu, The two-way random effects model and two-way mixed effects model are equivalent in their calculation and only differ in their interpretation. As such, this script provides a calculation of both.
I was wondering if there was a way to pick 2-way mixed or random model in this function?
I used 'nanmean' and 'nanstd' instead of mean_nan and std_nan and it worked fine
Great function, easy to use, and it also outputs the confidence interval and a reference for it! Excellent work!!
A phenomenal function for ICC--quick, painless, and complete. The only minor inconvenience is the fact that there are several functions that must be put into the code, but that is a small price to pay.
The uploaded .zip file was not valid in 1.3. Fixed the issue.
Removed the dependency to anova_rm function. Fixed a bug in computing the confidence intervals as pointed out by 'Erlend'.
Fixed the title
Fixed a bug in ICC.m that resulted in incorrect results
Inspired by: Repeated Measures ANOVA