Configure Incremental Learning Model
R2026bAn incremental learning model object fully specifies how functions implement incremental fitting and model performance evaluation. To configure (or prepare) an incremental learning model, create one by calling the object directly, or by converting a traditionally trained model to one of the objects. The following table lists the available model types, model objects for incremental learning, and conversion functions.
| Objective | Model Type | Model Object for Incremental Learning | Conversion Function |
|---|---|---|---|
| Binary classification | Linear support vector machine (SVM) and logistic regression with Gaussian kernels | incrementalClassificationKernel |
|
| Linear SVM and logistic regression | incrementalClassificationLinear |
| |
| Multiclass classification | Error-correcting output codes (ECOC) model with binary learners | incrementalClassificationECOC |
|
| Naive Bayes with normal, multinomial, or multivariate multinomial predictor conditional distributions | incrementalClassificationNaiveBayes |
| |
| Neural Network | incrementalClassificationNeuralNetwork | incrementalLearner converts a neural network classification model
(ClassificationNeuralNetwork). | |
| Regression | Least-squares and linear SVM regression with Gaussian kernels | incrementalRegressionKernel |
|
| Least-squares and linear SVM regression | incrementalRegressionLinear |
| |
| Neural Network | incrementalRegressionnNeuralNetwork | incrementalLearner converts a neural network regression model (RegressionNeuralNetwork). |
The approach you choose to create an incremental model depends on the information you have and your preferences.
Call object: Create an incremental model to your specifications by calling the object directly. This approach is flexible, enabling you to specify most options to suit your preferences, and the resulting model provides reasonable default values. For more details, see Call Object Directly.
Convert model: Convert a traditionally trained model to an incremental learner to initialize a model for incremental learning by using the
incrementalLearnerfunction. The function passes information that the traditionally trained model learned from the data. To convert a traditionally trained model, you must have a set of labeled data to which you can fit a model.When you use
incrementalLearner, you can specify all performance evaluation options and only those training, model, and data options that are unknown during conversion. For more details, see Convert Traditionally Trained Model.
Regardless of the approach you use, consider these configurations:
Model performance evaluation settings, such as the performance metrics to measure. For more details, see the individual object reference pages.
For ECOC models:
Binary learners
Coding design matrix for the binary learners.
For kernel models:
Model type, such as SVM
Objective function solver, such as standard stochastic gradient descent (SGD)
Hyperparameters for random feature expansion, such as the kernel scale parameter and number of dimensions of expanded space
For linear models:
Model type, such as SVM
Coefficient initial values
Objective function solver, such as standard stochastic gradient descent (SGD)
Solver hyperparameter values, such as the learning rate of SGD solvers
For neural network models:
Neural network layer parameters, such as weights, biases, and activations.
Objective function solver, such as FreeREX.
Solver hyperparameter values, such as the initial learning rate of the mini-batch LBFGS solver.
For naive Bayes models, the conditional distribution of the predictor variables. In a data set, you can specify that real-valued predictors are normally distributed and that categorical predictors (where levels are numeric scalars) are multivariate multinomial. For a bag-of-tokens model, where each predictor is a count, you can specify that all predictors are jointly multinomial.
Call Object Directly
Unlike when working with other machine learning model objects, you can create an incremental learning model by calling the corresponding object directly, with little knowledge about the data. For example, the following code creates a default incremental model for linear regression and a naive Bayes classification model for a data stream containing 5 classes.
MdlLR = incrementalRegressionLinear(); MdlNB = incrementalClassificationNaiveBayes(MaxNumClasses=5)
For linear and kernel models, the only information required to create a model directly is the machine learning problem, either classification or regression. An estimation period might also be required, depending on your specifications.
For naive Bayes and ECOC classification models, you must specify the maximum number of classes or all class names expected in the data during incremental learning.
For regression neural network models, no information is required.
For classification neural network models, you must specify one of the following:
Maximum number of classes expected in the data during incremental learning
Names of all expected classes
Weights, biases, and activations of the neural network layers
If you have information about the data to specify, or you want to configure model
options or performance evaluation settings, use name-value arguments when you call the
object. (All model properties are read-only; you cannot adjust them using dot notation.) For
example, the following pseudocode creates an incremental logistic regression model for
binary classification, initializes the linear model coefficients Beta and
bias Bias (obtained from prior knowledge of the problem), and sets the
performance metrics warm-up period to 500
observations.
Mdl = incrementalClassificationLinear(Learner="logistic", ... Beta=beta,Bias=bias,MetricsWarmupPeriod=500);
For more details on all options, see the Properties section of each incremental model object page.
Performance Evaluation Options and Properties
Performance evaluation properties and options enable you to configure how and when model performance is measured by the incremental learning function updateMetrics or updateMetricsAndFit. Regardless of the options you choose, first familiarize yourself with the incremental learning periods.
This table contains all performance evaluation options and properties.
| Performance Evaluation Options and Properties | Description |
|---|---|
Metrics | Specify the list of performance metrics or loss functions to measure
incrementally by using the Metrics name-value argument. The
Metrics property stores a table of tracked cumulative and
window metrics. |
MetricsWarmupPeiod | Number of observations to which the incremental model must be fit before it tracks performance metrics |
MetricsWindowSize | Number of observations to use to compute window performance metrics |
IsWarm*** | Flag indicating whether the model is warm (measures performance metrics) |
***You cannot specify the IsWarm property, whereas you can set the
other properties by using name-value argument syntax when you call the object.
The metrics specified by the Metrics name-value argument form a
table stored in the Metrics property of the model. For example, if you
specify Metrics=["Metric1","Metric2"] when you create an incremental
model Mdl, the Metrics property
is
>> Mdl.Metrics
ans =
2×2 table
Cumulative Window
__________ ______
Metric1 NaN NaN
Metric2 NaN NaN
Specify a positive metrics warm-up period when you believe the model is of low quality and needs to be trained before the function updateMetrics or updateMetricsAndFit tracks performance metrics in the Metrics property. In this case, the IsWarm property is false, and you must pass the incoming data and model to the incremental fitting function fit or updateMetricsAndFit.
When the incremental fitting function processes enough data to satisfy the estimation
period (for linear and kernel models), the solver turning period (for neural network
models using the mini-batch LBFGS solver) and the metrics warm-up period, the
IsWarm property becomes true, and you can measure
the model performance on incoming data and optionally train the model. For naive Bayes,
neural network, and ECOC classification models, incremental fitting functions must
additionally fit the model to all expected classes to become warm.
When the model is warm, updateMetrics or updateMetricsAndFit tracks all specified metrics cumulatively (from the start of the evaluation) and within a window of observations specified by the MetricsWindowSize property. Cumulative metrics reflect the model performance over the entire incremental learning history; after Performance Evaluation Period 1 starts, cumulative metrics are independent of the evaluation period. Window metrics reflect the model performance only over the specified window size for each performance evaluation period.
Convert Traditionally Trained Model
incrementalLearner enables you to initialize an incremental model
using information learned from a traditionally trained model. The converted model can
generate predictions and it is warm, which means that incremental learning functions can
measure model performance metrics from the start of the data stream. In other words,
estimation and performance metrics warm-up periods are not required for incremental
learning.
To convert a traditionally trained model to an incremental learner, pass the model and
any options specified by name-value arguments to incrementalLearner.
For example, the following pseudocode initializes an incremental classification model by
using all information that a linear SVM model for binary classification has learned from a
batch of
data.
Mdl = fitcsvm(X,Y); IncrementalMdl = incrementalLearner(Mdl,Name=Value);
IncrementalMdl is an incremental one-class SVM model object for
anomaly detection.
Ease of incremental model creation and initialization is offset by decreased
flexibility. The software assumes that fitted parameters, hyperparameter values, and data
characteristics learned during traditional training are appropriate for incremental
learning. Therefore, you cannot set corresponding learned or tuned options when you call
incrementalLearner.
The incrementalLearner function for each supported classification
and regression model transfers some properties from Mdl or infers from
other values. For more details, see the output argument description of each
incrementalLearner function page.
The following conditions apply when you convert a linear classification or regression
model (ClassificationLinear and RegressionLinear, respectively):
Incremental fitting functions support ridge (L2) regularization only.
Incremental fitting functions support the specification of only one regularization value. Therefore, if you specify a regularization path (vector of regularization values) when you call
fitclinearorfitrlinear, choose the model associated with one penalty by passing it toselectModels.If you solve the objective function by using standard or average SGD (
"sgd"or"asgd"for theSolvername-value argument), these conditions apply when you callincrementalLearner:incrementalLearnertransfers the solver used to optimizeMdltoIncrementalMdl.You can specify the adaptive scale-invariant solver
"scale-invariant"instead, but you cannot specify a different SGD solver.If you do not specify the adaptive scale-invariant solver,
incrementalLearnertransfers model and solver hyperparameter values to the incremental model object, such as the learning rateLearnRate, mini-batch sizeBatchSize, and ridge penaltyLambda. You cannot modify the transferred properties.
Call Object After Training Model
If you require more flexibility when you create an incremental model, you can call the object directly and initialize the model by individually setting learned information using name-value arguments. The following pseudocode show two examples:
Initialize an incremental classification model from the coefficients and class names learned by fitting a linear SVM model for binary classification to a batch of data
XcandYc.Mdl = fitcsvm(Xc,Yc); IncrementalMdl = incrementalClassificationLinear( ... Beta=Mdl.Beta,Bias=Mdl.Bias,ClassNames=Mdl.ClassNames);Initialize an incremental regression model from the coefficients learned by fitting a linear model to a batch of data
XrandYr.Mdl = fitlm(Xr,Yr); bias = Mdl.Coefficients.Estimate(1); beta = Mdl.Coefficients.Estimate(2:end); IncrementalMdl = incrementalRegressionLinear( ... Learner="leastsquares",Bias=bias,Beta=beta);
