Documentation

This is machine translation

Translated by
Mouseover text to see original. Click the button below to return to the English verison of the page.

CompactClassificationSVM class

Compact support vector machine for binary classification

Description

`CompactClassificationSVM` is a compact support vector machine (SVM) classifier.

The compact classifier does not include the data used for training the SVM classifier. Therefore, you cannot perform tasks, such as cross validation, using the compact classifier.

Use a compact SVM classifier for labeling new data (i.e., predicting the labels of new data).

Construction

```CompactSVMModel = compact(SVMModel)``` returns a compact SVM classifier (`CompactSVMModel`) from a full, trained support vector machine classifier (`SVMModel`).

Code Generation support: Yes.

MATLAB Function Block support: Yes.

Input Arguments

 `SVMModel` A full, trained `ClassificationSVM` classifier trained by `fitcsvm`.

Properties

`Alpha`

s-by-1 numeric vector of trained classifier coefficients from the dual problem, that is, the estimated Lagrange multipliers. s is the number of support vectors in the trained classifier, that is, `sum(Mdl.IsSupportVector)`.

If you specify removing duplicates using `RemoveDuplicates`, then for a given set of duplicate observations that are support vectors, `Alpha` contains one coefficient corresponding to the entire set. That is, MATLAB® attributes a nonzero coefficient to one observation from the set of duplicates and a coefficient of `0` to all other duplicate observations in the set.

`Beta`

Numeric vector of linear predictor coefficients. `Beta` has length equal to the number of predictors used to train the model.

If your predictor data contains categorical variables, then the software uses full dummy encoding for these variables. The software creates one dummy variable for each level of each categorical variable. `Beta` stores one value for each predictor variable, including the dummy variables. For example, if there are three predictors, one of which is a categorical variable with three levels, then `Beta` is a numeric vector containing five values.

If `KernelParameters.Function` is `'linear'`, then the software estimates the classification score for the observation x using

`$f\left(x\right)=\left(x/s\right)\prime \beta +b.$`

`Mdl` stores β, b, and s in the properties `Beta`, `Bias`, and `KernelParameters.Scale`, respectively.

If `KernelParameters.Function` is not `'linear'`, then `Beta` is empty (`[]`).

`Bias`

Scalar corresponding to the trained classifier bias term.

`CategoricalPredictors`

List of categorical predictors, which is always empty (`[]`) for SVM and discriminant analysis classifiers.

`ClassNames`

List of elements in `Y` with duplicates removed. `ClassNames` has the same data type as the data in the argument `Y`, and therefore can be a categorical or character array, logical or numeric vector, or cell array of character vectors.

`Cost`

Square matrix, where `Cost(i,j)` is the cost of classifying a point into class `j` if its true class is `i`.

During training, the software updates the prior probabilities by incorporating the penalties described in the cost matrix. Therefore,

• For two-class learning, `Cost` always has this form: `Cost(i,j) = 1` if `i ~= j`, and `Cost(i,j) = 0` if `i = j` (i.e., the rows correspond to the true class and the columns correspond to the predicted class). The order of the rows and columns of `Cost` corresponds to the order of the classes in `ClassNames`.

• For one-class learning, `Cost = 0`.

This property is read-only. For more details, see Algorithms.

`ExpandedPredictorNames`

Expanded predictor names, stored as a cell array of character vectors.

If the model uses encoding for categorical variables, then `ExpandedPredictorNames` includes the names that describe the expanded variables. Otherwise, `ExpandedPredictorNames` is the same as `PredictorNames`.

`KernelParameters`

Structure array containing the kernel name and parameter values.

To display the values of `KernelParameters`, use dot notation, e.g., `Mdl.KernelParameters.Scale` displays the scale parameter value.

The software accepts `KernelParameters` as inputs, and does not modify them. Alter `KernelParameters` by setting the appropriate name-value pair arguments when you train the SVM classifier using `fitcsvm`.

`Mu`

Numeric vector of predictor means.

If you specify `'Standardize',1` or `'Standardize',true` when you train an SVM classifier using `fitcsvm`, then `Mu` has length equal to the number of predictors.

If your predictor data contains categorical variables, then the software uses full dummy encoding for these variables. The software creates one dummy variable for each level of each categorical variable. `Mu` stores one value for each predictor variable, including the dummy variables. However, the software does not standardize the columns that contain categorical variables.

If `'Standardize'` is `false` or `0`, then `Mu` is an empty vector (`[]`).

`PredictorNames`

Cell array of character vectors containing the predictor names, in the order that they appear in the training data.

`Prior`

Numeric vector of prior probabilities for each class. The order of the elements of `Prior` corresponds to the elements of `Mdl.ClassNames`.

For two-class learning, if you specify a cost matrix, then the software updates the prior probabilities by incorporating the penalties described in the cost matrix.

This property is read-only. For more details, see Algorithms.

`ScoreTransform`

Character vector representing a built-in transformation function, or a function handle for transforming predicted classification scores.

To change the score transformation function to, e.g., `function`, use dot notation.

• For a built-in function, enter a character vector.

`Mdl.ScoreTransform = 'function';`

This table contains the available, built-in functions.

ValueFormula
`'doublelogit'`1/(1 + e–2x)
`'invlogit'`log(x / (1–x))
`'ismax'`Set the score for the class with the largest score to `1`, and scores for all other classes to `0`.
`'logit'`1/(1 + ex)
`'none'` or `'identity'`x (no transformation)
`'sign'`–1 for x < 0
0 for x = 0
1 for x > 0
`'symmetric'`2x – 1
`'symmetriclogit'`2/(1 + ex) – 1
`'symmetricismax'`Set the score for the class with the largest score to `1`, and scores for all other classes to `-1`.

• For a MATLAB function, or a function that you define, enter its function handle.

`Mdl.ScoreTransform = @function;`

`function` should accept a matrix (the original scores) and return a matrix of the same size (the transformed scores).

`Sigma`

Numeric vector of predictor standard deviations.

If you specify `'Standardize',1` or `'Standardize',true` when you train the SVM classifier, then `Sigma` has length equal to the number of predictors.

If your predictor data contains categorical variables, then the software uses full dummy encoding for these variables. The software creates one dummy variable for each level of each categorical variable. `Sigma` stores one value for each predictor variable, including the dummy variables. However, the software does not standardize the columns that contain categorical variables.

If `'Standardize'` is `false` or `0`, then `Sigma` is an empty vector (`[]`).

`SupportVectors`

s-by-p numeric matrix containing rows of `X` that MATLAB considers the support vectors. s is the number of support vectors in the trained classifier, that is, `sum(Mdl.IsSupportVector)` and p is the number of predictor variables in the predictor data.

If you specify `'Standardize',1` or `'Standardize',true`, then `SupportVectors` are the standardized rows of `X`.

If you specify removing duplicates using `RemoveDuplicates`, then for a given set of duplicate observations that are support vectors, `SupportVectors` contains one unique support vector.

`SupportVectorLabels`

s-by-1 numeric vector of support vector class labels. s is the number of support vectors in the trained classifier, that is, `sum(Mdl.IsSupportVector)`.

A value of `+1` indicates that the corresponding support vector is in the positive class (`Mdl.ClassNames{2}`). A value of `-1` indicates that the corresponding support vector is in the negative class (`Mdl.ClassNames{1}`).

If you specify removing duplicates using `RemoveDuplicates`, then for a given set of duplicate observations that are support vectors, `SupportVectorLabels` contains one unique support vector label.

Methods

 compareHoldout Compare accuracies of two classification models using new data discardSupportVectors Discard support vectors for linear support vector machine models edge Classification edge for support vector machine classifiers fitPosterior Fit posterior probabilities loss Classification error for support vector machine classifiers margin Classification margins for support vector machine classifiers predict Predict labels using support vector machine classification model

Copy Semantics

Value. To learn how value classes affect copy operations, see Copying Objects in the MATLAB documentation.

Examples

expand all

Full SVM classifiers (i.e., `ClassificationSVM` classifiers) hold the training data. For efficiency, you might not want to predict new labels using a large classifier. This example shows how to reduce the size of a full SVM classifier.

Load the `ionosphere` data set.

```load ionosphere ```

Train an SVM classifier. It is good practice to standardize the predictors and specify the order of the classes.

```SVMModel = fitcsvm(X,Y,'Standardize',true,... 'ClassNames',{'b','g'}) ```
```SVMModel = ClassificationSVM ResponseName: 'Y' CategoricalPredictors: [] ClassNames: {'b' 'g'} ScoreTransform: 'none' NumObservations: 351 Alpha: [89×1 double] Bias: -0.1341 KernelParameters: [1×1 struct] Mu: [1×34 double] Sigma: [1×34 double] BoxConstraints: [351×1 double] ConvergenceInfo: [1×1 struct] IsSupportVector: [351×1 logical] Solver: 'SMO' ```

`SVMModel` is a `ClassificationSVM` classifier.

Reduce the size of the SVM classifier.

```CompactSVMModel = compact(SVMModel) ```
```CompactSVMModel = classreg.learning.classif.CompactClassificationSVM ResponseName: 'Y' CategoricalPredictors: [] ClassNames: {'b' 'g'} ScoreTransform: 'none' Alpha: [89×1 double] Bias: -0.1341 KernelParameters: [1×1 struct] Mu: [1×34 double] Sigma: [1×34 double] SupportVectors: [89×34 double] SupportVectorLabels: [89×1 double] ```

`CompactSVMModel` is a `CompactClassificationSVM` classifier.

Display how much memory each classifier uses.

```whos('SVMModel','CompactSVMModel') ```
``` Name Size Bytes Class Attributes CompactSVMModel 1x1 30224 classreg.learning.classif.CompactClassificationSVM SVMModel 1x1 140546 ClassificationSVM ```

The full SVM classifier (`SVMModel`) is more than four times the compact SVM classifier (`CompactSVMModel`).

You can remove `SVMModel` from the MATLAB® Workspace, and pass `CompactSVMModel` and new predictor values to `predict` to efficiently label new observations.

Load the `ionosphere` data set.

```load ionosphere ```

Train and cross validate an SVM classifier. It is good practice to standardize the predictors and specify the order of the classes.

```rng(1); % For reproducibility CVSVMModel = fitcsvm(X,Y,'Standardize',true,... 'ClassNames',{'b','g'},'CrossVal','on') ```
```CVSVMModel = classreg.learning.partition.ClassificationPartitionedModel CrossValidatedModel: 'SVM' PredictorNames: {1×34 cell} ResponseName: 'Y' NumObservations: 351 KFold: 10 Partition: [1×1 cvpartition] ClassNames: {'b' 'g'} ScoreTransform: 'none' ```

`CVSVMModel` is not a `ClassificationSVM` classifier, but a `ClassificationPartitionedModel` cross-validated, SVM classifier. By default, the software implements 10-fold cross validation.

Alternatively, you can cross validate a trained `ClassificationSVM` classifier by passing it to `crossval`.

Inspect one of the trained folds using dot notation.

```CVSVMModel.Trained{1} ```
```ans = classreg.learning.classif.CompactClassificationSVM ResponseName: 'Y' CategoricalPredictors: [] ClassNames: {'b' 'g'} ScoreTransform: 'none' Alpha: [78×1 double] Bias: -0.2209 KernelParameters: [1×1 struct] Mu: [1×34 double] Sigma: [1×34 double] SupportVectors: [78×34 double] SupportVectorLabels: [78×1 double] ```

Each fold is a `CompactClassificationSVM` classifier trained on 90% of the data.

Estimate the generalization error.

```genError = kfoldLoss(CVSVMModel) ```
```genError = 0.1168 ```

On average, the generalization error is approximately 12%.

Extended Capabilities

Notes and limitations for code generation when training `SVMModel` include:

• The class labels input argument value (`Y`) cannot be a categorical array.

• The `ClassNames` name-value pair argument cannot be a categorical array.

• You cannot use the `CategoricalPredictors` name-value pair argument or supply a table containing at least one categorical predictor. That is, code generation does not support categorical predictors. To dummy-code variables that you want treated as categorical, use `dummyvar`.

• MATLAB does not support one-class learning.

• You cannot specify a score transformation function by using the `ScoreTransform` name-value pair argument or by assigning the `ScoreTransform` object property. Consequently, `saveCompactModel` cannot accept compact SVM models equipped to estimate class posterior probabilities, that is, models returned by `fitPosterior` or `fitSVMPosterior`.

MATLAB Function Block

You can use this function in the MATLAB Function Block in Simulink®.

References

[1] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, Second Edition. NY: Springer, 2008.

[2] Scholkopf, B., J. C. Platt, J. C. Shawe-Taylor, A. J. Smola, and R. C. Williamson. "Estimating the Support of a High-Dimensional Distribution." Neural Comput., Vol. 13, Number 7, 2001, pp. 1443–1471.

[3] Christianini, N., and J. C. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge, UK: Cambridge University Press, 2000.

[4] Scholkopf, B. and A. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond, Adaptive Computation and Machine Learning Cambridge, MA: The MIT Press, 2002.