Documentation |
Class: ClassificationSVM
Classification loss for support vector machine classifiers by resubstitution
L = resubLoss(SVMModel) returns the classification loss by resubstitution (L), the in-sample classification loss, for the support vector machine (SVM) classifier SVMModel using the training data stored in SVMModel.X and corresponding class labels stored in SVMModel.Y.
L = resubLoss(SVMModel,Name,Value) returns the classification loss by resubstitution with additional options specified by one or more Name,Value pair arguments.
The binomial deviance is a binary classification error measure that has the form
$$L=\frac{{\displaystyle \sum}_{j=1}^{n}{w}_{j}\mathrm{log}\left(1+\mathrm{exp}\left(-2{y}_{j}\prime f\left({X}_{j}\right)\right)\right)}{{\displaystyle \sum}_{j=1}^{n}{w}_{j}},$$
where:
w_{j} is weight j. The software renormalizes the weights to sum to 1.
y_{j} = {-1,1}.
$$f({X}_{j})$$ is the score for observation j.
The binomial deviance has connections to the maximization of the binomial likelihood function. For details on binomial deviance, see [1].
The classification error is a binary classification error measure that has the form
$$L=\frac{{\displaystyle \sum _{j=1}^{n}{w}_{j}{e}_{j}}}{{\displaystyle \sum _{j=1}^{n}{w}_{j}}},$$
where:
w_{j} is the weight for observation j. The software renormalizes the weights to sum to 1.
e_{j} = 1 if the predicted class of observation j differs from its true class, and 0 otherwise.
In other words, it is the proportion of observations that the classifier misclassifies.
A binary classification error measure that is similar to binomial deviance, and has the form
$$L=\frac{{\displaystyle \sum}_{j=1}^{n}{w}_{j}\mathrm{exp}\left(-{y}_{j}\prime f\left({X}_{j}\right)\right)}{{\displaystyle \sum}_{j=1}^{n}{w}_{j}},$$
where:
w_{j} is weight j. The software renormalizes the weights to sum to 1.
y_{j} = {-1,1}.
$$f({X}_{j})$$ is the score for observation j.
Hinge loss is a binary classification error measure that has the form
$$L=\frac{{\displaystyle \sum}_{j=1}^{n}{w}_{j}\mathrm{max}\left\{0,1-{y}_{j}\prime f\left({X}_{j}\right)\right\}}{{\displaystyle \sum}_{j=1}^{n}{w}_{j}},$$
where:
w_{j} is weight j. The software renormalizes the weights to sum to 1.
y_{j} = {-1,1}.
$$f({X}_{j})$$ is the score for observation j.
Hinge loss linearly penalizes for misclassified observations, and is related to the SVM objective function used for optimization. For more details on hinge loss, see [1].
The SVM score for classifying observation x is the signed distance from x to the decision boundary ranging from -∞ to +∞. A positive score for a class indicates that x is predicted to be in that class, a negative score indicates otherwise.
The score is also the numerical, predicted response for x, $$f(x)$$, computed by the trained SVM classification function
$$f(x)={\displaystyle \sum _{j=1}^{n}{\alpha}_{j}}{y}_{j}G({x}_{j},x)+b,$$
where $$({\alpha}_{1},\mathrm{...},{\alpha}_{n},b)$$ are the estimated SVM parameters, $$G({x}_{j},x)$$ is the dot product in the predictor space between x and the support vectors, and the sum includes the training set observations.
[1] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, second edition. Springer, New York, 2008.
ClassificationSVM | CompactClassificationSVM | fitcsvm | loss | resubMargin | resubPredict