A probabilistic model of classifier competence

Classifier competence based on full vector of class supports and probabilistic modelling.

In pattern recognition a common problem is to calculate competence of a classifier for a given object. Methods for calculating the competence currently developed are based only on crisp decision of the classifier, i.e. correct/incorrect classification.

The function ccprmod.m calculates the competence using full 1xC element vector of class supports produces by the classifier for the object, where C is the number of classes. The function is based on probabilistic modelling of class supports using C beta probability density functions (pdfs). First, parameters of the pdfs are defined in such a way that the expected value of each pdf is equal to the support given by the classifier for the respective class. A randomised reference classifier (RRC) is then constructed. The class supports of the RRC are random variables with the pdfs previously described. Finally, the classifier competence is calculated as the probability of correct classification of the RRC. For details, please see [1].

[1] Tomasz Woloszynski, Marek Kurzynski, A probabilistic model of classifier competence for dynamic ensemble selection, Pattern Recognition, Volume 44, Issues 10–11, October–November 2011, Pages 2656-2668