A set of MATLAB functions for evaluating generalization performance in binary classification.
acc_p(C)
% P value of the Null hypothesis that the accuracy is not significantly
% above chance (i.e., the area under the pdf below 0.5). This P value could
% then be compared to 0.05 or 0.025.
%
% Usage:
% a_p = acc_p(C)
%
% Arguments:
% C - 2x2 confusion matrix of classification outcomes
% Kay H. Brodersen, ETH Zurich, Switzerland
% http://people.inf.ethz.ch/bkay/
% $Id: acc_p.m 8245 2010-10-22 12:57:51Z bkay $
% -------------------------------------------------------------------------
function a_p = acc_p(C)
% Get alpha and beta
A = C(1,1) + C(2,2) + 1;
B = C(1,2) + C(2,1) + 1;
% Compute area under pdf below 0.5
a_p = betacdf(0.5,A,B);
end