resubLoss

Class: ClassificationSVM

Classification loss for support vector machine classifiers by resubstitution

Syntax

  • L = resubLoss(SVMModel) example
  • L = resubLoss(SVMModel,Name,Value) example

Description

example

L = resubLoss(SVMModel) returns the classification loss by resubstitution (L), the in-sample classification loss, for the support vector machine (SVM) classifier SVMModel using the training data stored in SVMModel.X and corresponding class labels stored in SVMModel.Y.

example

L = resubLoss(SVMModel,Name,Value) returns the classification loss by resubstitution with additional options specified by one or more Name,Value pair arguments.

Input Arguments

expand all

SVMModel — Full, trained SVM classifierClassificationSVM classifier

Full, trained SVM classifier, specified as a ClassificationSVM model trained using fitcsvm.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside single quotes (' '). You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

'LossFun' — Loss function'ClassifError' (default) | 'binodeviance' | 'exponential' | 'hinge' | function handle

Loss function, specified as the comma-separated pair consisting of 'LossFun' and a function handle or string.

  • The following lists available loss functions. Specify one using its corresponding string.

    ValueLoss Function
    'binodeviance'Binomial deviance
    'classiferror'Classification error
    'exponential'Exponential loss
    'hinge'Hinge loss

  • Specify your own function using function handle notation.

    Suppose that n = size(X,1) is the sample size and k = size(SVMModel.ClassNames,1) is the number of classes. Your function must have the signature lossvalue = lossfun(C,S,W,Cost), where:

    • The output argument lossvalue is a scalar.

    • You choose the function name (lossfun).

    • C is an n-by-k logical matrix with rows indicating which class the corresponding observation belongs. The column order corresponds to the class order in SVMModel.ClassNames.

      Construct C by setting C(p,q) = 1 if observation p is in class q, for each row. Set all other elements of row p to 0.

    • S is an n-by-k numeric matrix of classification scores. The column order corresponds to the class order in SVMModel.ClassNames. S is a matrix of classification scores, similar to the output of predict.

    • W is an n-by-1 numeric vector of observation weights. If you pass W, the software normalizes them to sum to 1.

    • Cost is a k-by-k numeric matrix of misclassification costs. For example, Cost = ones(K) - eye(K) specifies a cost of 0 for correct classification, and 1 for misclassification.

    Specify your function using 'LossFun',@lossfun.

Output Arguments

expand all

L — Classification lossscalar

Classification loss, returned as a scalar. L is a generalization or resubstitution quality measure. Its interpretation depends on the loss function and weighting scheme, but, in general, better classifiers yield smaller loss values.

Definitions

Binomial Deviance

The binomial deviance is a binary classification error measure that has the form

L=j=1nwjlog(1+exp(2yjf(Xj)))j=1nwj,

where:

  • wj is weight j. The software renormalizes the weights to sum to 1.

  • yj = {-1,1}.

  • f(Xj) is the score for observation j.

The binomial deviance has connections to the maximization of the binomial likelihood function. For details on binomial deviance, see [1].

Classification Error

The classification error is a binary classification error measure that has the form

L=j=1nwjejj=1nwj,

where:

  • wj is the weight for observation j. The software renormalizes the weights to sum to 1.

  • ej = 1 if the predicted class of observation j differs from its true class, and 0 otherwise.

In other words, it is the proportion of observations that the classifier misclassifies.

Exponential Loss

A binary classification error measure that is similar to binomial deviance, and has the form

L=j=1nwjexp(yjf(Xj))j=1nwj,

where:

  • wj is weight j. The software renormalizes the weights to sum to 1.

  • yj = {-1,1}.

  • f(Xj) is the score for observation j.

Hinge Loss

Hinge loss is a binary classification error measure that has the form

L=j=1nwjmax{0,1yjf(Xj)}j=1nwj,

where:

  • wj is weight j. The software renormalizes the weights to sum to 1.

  • yj = {-1,1}.

  • f(Xj) is the score for observation j.

Hinge loss linearly penalizes for misclassified observations, and is related to the SVM objective function used for optimization. For more details on hinge loss, see [1].

Score

The SVM score for classifying observation x is the signed distance from x to the decision boundary ranging from -∞ to +∞. A positive score for a class indicates that x is predicted to be in that class, a negative score indicates otherwise.

The score is also the numerical, predicted response for x, f(x), computed by the trained SVM classification function

f(x)=j=1nαjyjG(xj,x)+b,

where (α1,...,αn,b) are the estimated SVM parameters, G(xj,x) is the dot product in the predictor space between x and the support vectors, and the sum includes the training set observations.

Examples

expand all

Determine the Resubstitution Loss of SVM Classifiers

Load the ionosphere data set.

load ionosphere

Train an SVM classifier. It is good practice to standardize the data.

SVMModel = fitcsvm(X,Y,'ClassNames',{'b','g'},'Standardize',true);

SVMModel is a trained ClassificationSVM classifier. The negative class is 'b' and the positive class is 'g'.

Estimate the resubstitution loss (i.e., the in-sample classification error).

L = resubLoss(SVMModel)
L =

    0.0570

The SVM classifier misclassifies 5.7% of the training sample radar returns.

Determine the Resubstitution Hinge Loss of SVM Classifiers

Load the ionosphere data set.

load ionosphere

Train an SVM classifier. It is good practice to standardize the data.

SVMModel = fitcsvm(X,Y,'ClassNames',{'b','g'},'Standardize',true);

SVMModel is a trained ClassificationSVM classifier. The negative class is 'b' and the positive class is 'g'.

Estimate the in-sample hinge loss.

L = resubLoss(SVMModel,'LossFun','Hinge')
L =

    0.1603

The hinge loss is 0.1603. Classifiers with hinge losses close to 0 are desirable.

References

[1] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, second edition. Springer, New York, 2008.

Was this topic helpful?