Class: ClassificationPartitionedModel
Classification loss for observations not used for training
L = kfoldLoss(obj)
L = kfoldLoss(obj,Name,Value)
returns
loss obtained by crossvalidated classification model L
= kfoldLoss(obj
)obj
.
For every fold, this method computes classification loss for infold
observations using a model trained on outoffold observations.
calculates
loss with additional options specified by one or more L
= kfoldLoss(obj
,Name,Value
)Name,Value
pair
arguments. You can specify several namevalue pair arguments in any
order as Name1,Value1,…,NameN,ValueN
.

Object of class 
Specify optional commaseparated pairs of Name,Value
arguments.
Name
is the argument
name and Value
is the corresponding
value. Name
must appear
inside single quotes (' '
).
You can specify several name and value pair
arguments in any order as Name1,Value1,...,NameN,ValueN
.

Indices of folds ranging from Default:  

Loss function, specified as the commaseparated pair consisting
of
For more details on loss functions, see Classification Loss. Default:  

A character vector for determining the output of
Default: 

Loss, by default the fraction of misclassified data. 
Classification loss functions measure the predictive inaccuracy of classification models. When comparing the same type of loss among many models, lower loss indicates a better predictive model.
Suppose that:
L is the weighted average classification loss.
n is the sample size.
For binary classification:
y_{j} is the observed class label. The software codes it as –1 or 1 indicating the negative or positive class, respectively.
f(X_{j}) is the raw classification score for observation (row) j of the predictor data X.
m_{j} = y_{j}f(X_{j}) is the classification score for classifying observation j into the class corresponding to y_{j}. Positive values of m_{j} indicate correct classification and do not contribute much to the average loss. Negative values of m_{j} indicate incorrect classification and contribute to the average loss.
For algorithms that support multiclass classification (that is, K ≥ 3):
y_{j}^{*} is
a vector of K – 1 zeros, and a 1 in the
position corresponding to the true, observed class y_{j}.
For example, if the true class of the second observation is the third
class and K = 4, then y^{*}_{2} =
[0 0 1 0]′. The order of the classes corresponds to the order
in the ClassNames
property of the input model.
f(X_{j})
is the length K vector of class scores for observation j of
the predictor data X. The order of the scores corresponds
to the order of the classes in the ClassNames
property
of the input model.
m_{j} = y_{j}^{*}′f(X_{j}). Therefore, m_{j} is the scalar classification score that the model predicts for the true, observed class.
The weight for observation j is w_{j}. The software normalizes the observation weights so that they sum to the corresponding prior class probability. The software also normalizes the prior probabilities so they sum to 1. Therefore,
$$\sum _{j=1}^{n}{w}_{j}}=1.$$
The supported loss functions are:
Binomial deviance, specified using 'LossFun','binodeviance'
.
Its equation is
$$L={\displaystyle \sum _{j=1}^{n}{w}_{j}\mathrm{log}\left\{1+\mathrm{exp}\left[2{m}_{j}\right]\right\}}.$$
Exponential loss, specified using 'LossFun','exponential'
.
Its equation is
$$L={\displaystyle \sum _{j=1}^{n}{w}_{j}\mathrm{exp}\left({m}_{j}\right)}.$$
Classification error, specified using 'LossFun','classiferror'
.
It is the weighted fraction of misclassified observations, with equation
$$L={\displaystyle \sum _{j=1}^{n}{w}_{j}}I\left\{{\widehat{y}}_{j}\ne {y}_{j}\right\}.$$
$${\widehat{y}}_{j}$$ is the class label corresponding to the class with the maximal posterior probability. I{x} is the indicator function.
Hinge loss, specified using 'LossFun','hinge'
.
Its equation is
$$L={\displaystyle \sum}_{j=1}^{n}{w}_{j}\mathrm{max}\left\{0,1{m}_{j}\right\}.$$
Logit loss, specified using 'LossFun','logit'
.
Its equation is
$$L={\displaystyle \sum _{j=1}^{n}{w}_{j}\mathrm{log}\left(1+\mathrm{exp}\left({m}_{j}\right)\right)}.$$
Minimal cost, specified using 'LossFun','mincost'
.
The software computes the weighted minimal cost using this procedure
for observations j = 1,...,n:
Estimate the 1byK vector of expected classification costs for observation j
$${\gamma}_{j}=f{\left({X}_{j}\right)}^{\prime}C.$$
f(X_{j})
is the column vector of class posterior probabilities for binary and
multiclass classification. C is the cost matrix
the input model stores in the property Cost
.
For observation j, predict the class label corresponding to the minimum, expected classification cost:
$${\widehat{y}}_{j}=\underset{j=1,\mathrm{...},K}{\mathrm{min}}\left({\gamma}_{j}\right).$$
Using C, identify the cost incurred (c_{j}) for making the prediction.
The weighted, average, minimum cost loss is
$$L={\displaystyle \sum _{j=1}^{n}{w}_{j}{c}_{j}}.$$
Quadratic loss, specified using 'LossFun','quadratic'
.
Its equation is
$$L={\displaystyle \sum _{j=1}^{n}{w}_{j}{\left(1{m}_{j}\right)}^{2}}.$$
This figure compares some of the loss functions for one observation over m (some functions are normalized to pass through [0,1]).
ClassificationPartitionedModel
 crossval
 kfoldEdge
 kfoldfun
 kfoldMargin
 kfoldPredict