Code covered by the BSD License
 acc_cmp1(targs,preds1,pre...Tests the null hypothesis that two classifiers that were tested on
 acc_cmp2(C1,C2)Tests the null hypothesis that two classifiers that were tested on
 acc_mean(C)Expected value of the accuracy (i.e., the mean of a Beta distribution).
 acc_med(C)Median of the accuracy (i.e., the median of a Beta distribution).
 acc_mode(C)Most likely accuracy (i.e., the mode of a Beta distribution)
 acc_p(C)P value of the Null hypothesis that the accuracy is not significantly
 acc_ppi(C,alpha)Posterior probability interval of the accuracy.
 acc_sem(C)Naive standarderrorofthemean of the acrosstestcases mean accuracy.
 adjustErrorBarWidth(hE, w...Adjusts error bar widths. Use in conjunction with errorbar().
 bacc_demoSimple demo to compare accuracies and balanced accuracies.
 bacc_mean(C)Expected value of the balanced accuracy (i.e., the mean of the average of
 bacc_med(C)Median of the balanced accuracy (i.e., the median of the average of
 bacc_mode(C)Most likely balanced accuracy (i.e., the mode of the average of
 bacc_naive(C)Naive balanced accuracy (simply the mean of the individual accuracies,
 bacc_p(C)P value of the Null hypothesis that the balanced accuracy is not
 bacc_ppi(C,alpha)Posterior probability interval of the balanced accuracy.
 betaavgcdf(x, alpha1, bet...CDF of the average of two independent random variables which are
 betaavginv(y, alpha1, bet...Inverse CDF of the sum of two independent random variables which are
 betaavgpdf(x, alpha1, bet...PDF of the average of two independent random variables which are
 betaconv(res, alpha1, bet...Convolves two Beta distributions.
 betasumcdf(x, alpha1, bet...CDF of the sum of two independent random variables which are distributed
 betasumpdf(x, alpha1, bet...PDF of the sum of two independently distributed Beta distributions.

View all files
Computing the posterior balanced accuracy
by
Kay H. Brodersen
02 Nov 2010
A set of MATLAB functions for evaluating generalization performance in binary classification.

acc_p(C) 
% P value of the Null hypothesis that the accuracy is not significantly
% above chance (i.e., the area under the pdf below 0.5). This P value could
% then be compared to 0.05 or 0.025.
%
% Usage:
% a_p = acc_p(C)
%
% Arguments:
% C  2x2 confusion matrix of classification outcomes
% Kay H. Brodersen, ETH Zurich, Switzerland
% http://people.inf.ethz.ch/bkay/
% $Id: acc_p.m 8245 20101022 12:57:51Z bkay $
% 
function a_p = acc_p(C)
% Get alpha and beta
A = C(1,1) + C(2,2) + 1;
B = C(1,2) + C(2,1) + 1;
% Compute area under pdf below 0.5
a_p = betacdf(0.5,A,B);
end


Contact us