There are two costs associated with KNN classification: the
true misclassification cost per class, and the expected misclassification
cost per observation. The third output of predict
is
the expected misclassification cost per observation.
Suppose you have Nobs
observations that you
want to classify with a trained classifier mdl
.
Suppose you have K
classes. You place the observations
into a matrix Xnew
with one observation per row.
The command
[label,score,cost] = predict(mdl,Xnew)
returns, among other outputs, a cost
matrix
of size Nobs
-by-K
. Each row
of the cost
matrix contains the expected (average)
cost of classifying the observation into each of the K
classes. cost(n,k)
is
$$\sum _{i=1}^{K}\widehat{P}\left(i|Xnew(n)\right)C\left(k|i\right)},$$
where
K is the number of classes.
$$\widehat{P}\left(i|Xnew(n)\right)$$ is the posterior probability of
class i for observation Xnew(n).
$$C\left(k|i\right)$$ is the true misclassification
cost of classifying an observation as k when
its true class is i.