Create generalized linear regression model by stepwise regression
creates a generalized linear model of a table or dataset array
mdl = stepwiseglm(
tbl using stepwise regression to add or remove predictors,
starting from a constant model.
stepwiseglm uses the last
tbl as the response variable.
stepwiseglm uses forward and backward stepwise regression
to determine a final model. At each step, the function searches for terms to add the
model to or remove from the model, based on the value of the
specifies additional options using one or more name-value pair arguments. For
example, you can specify the categorical variables, the smallest or largest set of
terms to use in the model, the maximum number of steps to take, or the criterion
mdl = stepwiseglm(___,
stepwiseglm uses to add or remove terms.
Create response data using just three of 20 predictors, and create a generalized linear model using stepwise algorithm to see if it uses just the correct predictors.
Create data with 20 predictors, and Poisson response using just three of the predictors, plus a constant.
rng('default') % for reproducibility X = randn(100,20); mu = exp(X(:,[5 10 15])*[.4;.2;.3] + 1); y = poissrnd(mu);
Fit a generalized linear model using the Poisson distribution.
mdl = stepwiseglm(X,y,... 'constant','upper','linear','Distribution','poisson')
1. Adding x5, Deviance = 134.439, Chi2Stat = 52.24814, PValue = 4.891229e-13 2. Adding x15, Deviance = 106.285, Chi2Stat = 28.15393, PValue = 1.1204e-07 3. Adding x10, Deviance = 95.0207, Chi2Stat = 11.2644, PValue = 0.000790094
mdl = Generalized linear regression model: log(y) ~ 1 + x5 + x10 + x15 Distribution = Poisson Estimated Coefficients: Estimate SE tStat pValue ________ ________ ______ __________ (Intercept) 1.0115 0.064275 15.737 8.4217e-56 x5 0.39508 0.066665 5.9263 3.0977e-09 x10 0.18863 0.05534 3.4085 0.0006532 x15 0.29295 0.053269 5.4995 3.8089e-08 100 observations, 96 error degrees of freedom Dispersion: 1 Chi^2-statistic vs. constant model: 91.7, p-value = 9.61e-20
The starting model is the constant model.
stepwiseglm by default uses deviance of the model as the criterion. It first adds
x5 into the model, as the -value for the test statistic, deviance (the differences in the deviances of the two models), is less than the default threshold value 0.05. Then, it adds
x15 because given
x5 is in the model, when
x15 is added, the -value for chi-squared test is smaller than 0.05. It then adds
x10 because given
x15 are in the model, when
x10 is added, the -value for the chi-square test statistic is again less than 0.05.
modelspec— Starting model
'constant'(default) | character vector or string scalar naming the model | t-by-(p + 1) terms matrix | character vector or string scalar formula in the form
'Y ~ terms'
Starting model for
as one of the following:
A character vector or string scalar naming the model.
|Model contains only a constant (intercept) term.|
|Model contains an intercept and linear term for each predictor.|
|Model contains an intercept, linear term for each predictor, and all products of pairs of distinct predictors (no squared terms).|
|Model contains an intercept term and linear and squared terms for each predictor.|
|Model contains an intercept term, linear and squared terms for each predictor, and all products of pairs of distinct predictors.|
|Model is a polynomial with all terms up to degree |
A t-by-(p + 1) matrix, or a Terms Matrix, specifying terms in the model, where t is the number of terms and p is the number of predictor variables, and +1 accounts for the response variable. A terms matrix is convenient when the number of predictors is large and you want to generate the terms programmatically.
A character vector or string scalar representing a Formula in the form
'Y ~ terms',
terms are in Wilkinson Notation. The variable names in the
formula must be valid MATLAB identifiers.
comma-separated pairs of
the argument name and
Value is the corresponding value.
Name must appear inside quotes. You can specify several name and value
pair arguments in any order as
'Criterion','aic','Distribution','poisson','Upper','interactions'specifies Akaike Information Criterion as the criterion to add or remove variables to the model, Poisson distribution as the distribution of the response variable, and a model with all possible interactions as the largest model to consider as the fit.
'Criterion'— Criterion to add or remove terms
Criterion to add or remove terms, specified as the comma-separated pair consisting of
'Criterion' and one of the following:
'Deviance' — p-value for F or
chi-squared test of the change in the deviance by adding or removing the term.
F-test is for testing a single model. Chi-squared test is
for comparing two different models.
'sse' — p-value for an F-test of the
change in the sum of squared error by adding or removing the term.
'aic' — Change in the value of Akaike information criterion (AIC).
'bic' — Change in the value of Bayesian information criterion (BIC).
'rsquared' — Increase in the value of R2.
'adjrsquared' — Increase in the value of adjusted R2.
'PEnter'— Threshold for criterion to add term
Threshold for the criterion to add a term, specified as the comma-separated pair
'PEnter' and a scalar value, as described in this
|0.05||If the p-value of F-statistic
or chi-squared statistic is less than |
|0.05||If the SSE of the model is less than |
|0||If the change in the AIC of the model is less than
|0||If the change in the BIC of the model is less than
|0.1||If the increase in the R-squared value of the model is greater than
|0||If the increase in the adjusted R-squared value of the model is
greater than |
For more information, see the
Criterion name-value pair
'PRemove'— Threshold for criterion to remove term
Threshold for the criterion to remove a term, specified as the comma-separated pair
'PRemove' and a scalar value, as described in this
|0.10||If the p-value of F-statistic
or chi-squared statistic is greater than |
|0.10||If the p-value of the F statistic is greater than
|0.01||If the change in the AIC of the model is greater than
|0.01||If the change in the BIC of the model is greater than
|0.05||If the increase in the R-squared value of the model is less than
|-0.05||If the increase in the adjusted R-squared value of the model is less
At each step, the
stepwiseglm function also checks whether a term
is redundant (linearly dependent) with other terms in the current model. When any term
is linearly dependent with other terms in the current model, the
stepwiseglm function removes the redundant term, regardless of
the criterion value.
For more information, see the
Criterion name-value pair
The generalized linear model
a standard linear model unless you specify otherwise with the
For other methods such as
devianceTest, or properties of the
GeneralizedLinearModel object, see
After training a model, you can generate C/C++ code that predicts responses for new data. Generating C/C++ code requires MATLAB Coder™. For details, see Introduction to Code Generation.
Stepwise regression is a systematic method
for adding and removing terms from a linear or generalized linear
model based on their statistical significance in explaining the response
variable. The method begins with an initial model, specified using
and then compares the explanatory power of incrementally larger and
stepwiseglm function uses forward and backward stepwise regression to
determine a final model. At each step, the function searches for terms to add to the model
or remove from the model based on the value of the
The default value of
'Criterion' for a linear regression model is
'sse'. In this case,
LinearModel use the
p-value of an F-statistic to test models with and
without a potential term at each step. If a term is not currently in the model, the null
hypothesis is that the term would have a zero coefficient if added to the model. If there is
sufficient evidence to reject the null hypothesis, the function adds the term to the model.
Conversely, if a term is currently in the model, the null hypothesis is that the term has a
zero coefficient. If there is insufficient evidence to reject the null hypothesis, the
function removes the term from the model.
Stepwise regression takes these steps when
Fit the initial model.
Examine a set of available terms not in the model. If any of the terms have p-values less than an entrance tolerance (that is, if it is unlikely a term would have a zero coefficient if added to the model), add the term with the smallest p-value and repeat this step; otherwise, go to step 3.
If any of the available terms in the model have p-values greater than an exit tolerance (that is, the hypothesis of a zero coefficient cannot be rejected), remove the term with the largest p-value and return to step 2; otherwise, end the process.
At any stage, the function will not add a higher-order term if the model does not also include
all lower-order terms that are subsets of the higher-order term. For example, the function
will not try to add the term
X1:X2^2 unless both
X2^2 are already in the model. Similarly, the function will not
remove lower-order terms that are subsets of higher-order terms that remain in the model.
For example, the function will not try to remove
X1:X2^2 remains in the model.
You can specify other criteria by using the
'Criterion' name-value pair
argument. For example, you can specify the change in the value of the Akaike information
criterion, Bayesian information criterion, R-squared, or adjusted R-squared as the criterion
to add or remove terms.
Depending on the terms included in the initial model, and the order in which the function adds and removes terms, the function might build different models from the same set of potential terms. The function terminates when no single step improves the model. However, a different initial model or a different sequence of steps does not guarantee a better fit. In this sense, stepwise models are locally optimal, but might not be globally optimal.
stepwiseglm treats a categorical predictor as follows:
A model with a categorical predictor that has L levels
(categories) includes L – 1 indicator variables. The model uses the first category as a
reference level, so it does not include the indicator variable for the reference
level. If the data type of the categorical predictor is
categorical, then you can check the order of categories
categories and reorder the
categories by using
reordercats to customize the
stepwiseglm treats the group of L – 1 indicator variables as a single variable. If you want to treat
the indicator variables as distinct predictor variables, create indicator
variables manually by using
dummyvar. Then use the
indicator variables, except the one corresponding to the reference level of the
categorical variable, when you fit a model. For the categorical predictor
X, if you specify all columns of
dummyvar(X) and an intercept term as predictors, then the
design matrix becomes rank deficient.
Interaction terms between a continuous predictor and a categorical predictor with L levels consist of the element-wise product of the L – 1 indicator variables with the continuous predictor.
Interaction terms between two categorical predictors with L and M levels consist of the (L – 1)*(M – 1) indicator variables to include all possible combinations of the two categorical predictor levels.
You cannot specify higher-order terms for a categorical predictor because the square of an indicator is equal to itself.
stepwiseglm adds or removes
a categorical predictor, the function actually adds or removes the group of indicator variables
in one step. Similarly, if
stepwiseglm adds or removes an interaction term
with a categorical predictor, the function actually adds or removes the group of interaction
terms including the categorical predictor.
'' (empty character vector),
"" (empty string),
<undefined> values in
Y to be missing values.
stepwiseglm does not use observations with missing values in the fit.
ObservationInfo property of a fitted model indicates whether or not
stepwiseglm uses each observation in the fit.
 Collett, D. Modeling Binary Data. New York: Chapman & Hall, 2002.
 Dobson, A. J. An Introduction to Generalized Linear Models. New York: Chapman & Hall, 1990.
 McCullagh, P., and J. A. Nelder. Generalized Linear Models. New York: Chapman & Hall, 1990.