GeneralizedLinearModel.stepwise

Class: GeneralizedLinearModel

(Not Recommended) Create generalized linear regression model by stepwise regression

GeneralizedLinearModel.stepwise is not recommended. Use stepwiseglm instead.

Syntax

mdl = GeneralizedLinearModel.stepwise(tbl,modelspec)
mdl = GeneralizedLinearModel.stepwise(X,y,modelspec)
mdl = GeneralizedLinearModel.stepwise(...,modelspec,Name,Value)

Description

mdl = GeneralizedLinearModel.stepwise(tbl,modelspec) creates a generalized linear model of a table or dataset array tbl, using stepwise regression to add or remove predictors. modelspec is the starting model for the stepwise procedure.

mdl = GeneralizedLinearModel.stepwise(X,y,modelspec) creates a generalized linear model of the responses y to a data matrix X, using stepwise regression to add or remove predictors.

mdl = GeneralizedLinearModel.stepwise(...,modelspec,Name,Value) creates a generalized linear model with additional options specified by one or more Name,Value pair arguments.

Input Arguments

expand all

Input data including predictor and response variables, specified as a table or dataset array. The predictor variables and response variable can be numeric, logical, categorical, character, or string. The response variable can have a data type other than numeric only if 'Distribution' is 'binomial'.

  • By default, GeneralizedLinearModel.stepwise takes the last variable as the response variable and the others as the predictor variables.

  • To set a different column as the response variable, use the ResponseVar name-value pair argument.

  • To use a subset of the columns as predictors, use the PredictorVars name-value pair argument.

  • To define a model specification, set the modelspec argument using a formula or terms matrix. The formula or terms matrix specifies which columns to use as the predictor or response variables.

The variable names in a table do not have to be valid MATLAB® identifiers. However, if the names are not valid, you cannot use a formula when you fit or adjust a model; for example:

  • You cannot specify modelspec using a formula.

  • You cannot use a formula to specify the terms to add or remove when you use the addTerms function or the removeTerms function, respectively.

  • You cannot use a formula to specify the lower and upper bounds of the model when you use the step or stepwiseglm function with the name-value pair arguments 'Lower' and 'Upper', respectively.

You can verify the variable names in tbl by using the isvarname function. The following code returns logical 1 (true) for each variable that has a valid variable name.

cellfun(@isvarname,tbl.Properties.VariableNames)
If the variable names in tbl are not valid, then convert them by using the matlab.lang.makeValidName function.
tbl.Properties.VariableNames = matlab.lang.makeValidName(tbl.Properties.VariableNames);

Predictor variables, specified as an n-by-p matrix, where n is the number of observations and p is the number of predictor variables. Each column of X represents one variable, and each row represents one observation.

By default, there is a constant term in the model, unless you explicitly remove it, so do not include a column of 1s in X.

Data Types: single | double

Response variable, specified as a vector or matrix.

  • If 'Distribution' is not 'binomial', then y must be an n-by-1 vector, where n is the number of observations. Each entry in y is the response for the corresponding row of X. The data type must be single or double.

  • If 'Distribution' is 'binomial', then y can be an n-by-1 vector or n-by-2 matrix with counts in column 1 and BinomialSize in column 2.

Data Types: single | double | logical | categorical

Starting model, specified as one of the following:

  • Character vector or string scalar specifying the type of model.

    ValueModel Type
    'constant'Model contains only a constant (intercept) term.
    'linear'Model contains an intercept and linear term for each predictor.
    'interactions'Model contains an intercept, linear term for each predictor, and all products of pairs of distinct predictors (no squared terms).
    'purequadratic'Model contains an intercept term and linear and squared terms for each predictor.
    'quadratic'Model contains an intercept term, linear and squared terms for each predictor, and all products of pairs of distinct predictors.
    'polyijk'Model is a polynomial with all terms up to degree i in the first predictor, degree j in the second predictor, and so on. Specify the maximum degree for each predictor by using numerals 0 though 9. The model contains interaction terms, but the degree of each interaction term does not exceed the maximum value of the specified degrees. For example, 'poly13' has an intercept and x1, x2, x22, x23, x1*x2, and x1*x22 terms, where x1 and x2 are the first and second predictors, respectively.
  • t-by-(p+1) matrix, namely terms matrix, specifying terms to include in model, where t is the number of terms and p is the number of predictor variables, and plus one is for the response variable.

  • Character vector or string scalar representing a formula in the form

    'Y ~ terms',

    where the terms are in Wilkinson Notation.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Number of trials for binomial distribution, that is the sample size, specified as the comma-separated pair consisting of 'BinomialSize' and the variable name in tbl, a numeric scalar, or a numeric vector of the same length as the response. This is the parameter n for the fitted binomial distribution. BinomialSize applies only when the Distribution parameter is 'binomial'.

If BinomialSize is a scalar value, that means all observations have the same number of trials.

As an alternative to BinomialSize, you can specify the response as a two-column matrix with counts in column 1 and BinomialSize in column 2.

Data Types: single | double | char | string

Categorical variable list, specified as the comma-separated pair consisting of 'CategoricalVars' and either a string array or cell array of character vectors containing categorical variable names in the table or dataset array tbl, or a logical or numeric index vector indicating which columns are categorical.

  • If data is in a table or dataset array tbl, then, by default, GeneralizedLinearModel.stepwise treats all categorical values, logical values, character arrays, string arrays, and cell arrays of character vectors as categorical variables.

  • If data is in matrix X, then the default value of 'CategoricalVars' is an empty matrix []. That is, no variable is categorical unless you specify it as categorical.

For example, you can specify the observations 2 and 3 out of 6 as categorical using either of the following examples.

Example: 'CategoricalVars',[2,3]

Example: 'CategoricalVars',logical([0 1 1 0 0 0])

Data Types: single | double | logical | string | cell

Criterion to add or remove terms, specified as the comma-separated pair consisting of 'Criterion' and one of the following:

  • 'Deviance'p-value for F or chi-squared test of the change in the deviance by adding or removing the term. F-test is for testing a single model. Chi-squared test is for comparing two different models.

  • 'sse'p-value for an F-test of the change in the sum of squared error by adding or removing the term.

  • 'aic' — Change in the value of Akaike information criterion (AIC).

  • 'bic' — Change in the value of Bayesian information criterion (BIC).

  • 'rsquared' — Increase in the value of R2.

  • 'adjrsquared' — Increase in the value of adjusted R2.

Example: 'Criterion','bic'

Indicator to compute dispersion parameter for 'binomial' and 'poisson' distributions, specified as the comma-separated pair consisting of 'DispersionFlag' and one of the following.

trueEstimate a dispersion parameter when computing standard errors
falseDefault. Use the theoretical value when computing standard errors

The fitting function always estimates the dispersion for other distributions.

Example: 'DispersionFlag',true

Distribution of the response variable, specified as the comma-separated pair consisting of 'Distribution' and one of the following.

'normal'Normal distribution
'binomial'Binomial distribution
'poisson'Poisson distribution
'gamma'Gamma distribution
'inverse gaussian'Inverse Gaussian distribution

Example: 'Distribution','gamma'

Observations to exclude from the fit, specified as the comma-separated pair consisting of 'Exclude' and a logical or numeric index vector indicating which observations to exclude from the fit.

For example, you can exclude observations 2 and 3 out of 6 using either of the following examples.

Example: 'Exclude',[2,3]

Example: 'Exclude',logical([0 1 1 0 0 0])

Data Types: single | double | logical

Indicator for the constant term (intercept) in the fit, specified as the comma-separated pair consisting of 'Intercept' and either true to include or false to remove the constant term from the model.

Use 'Intercept' only when specifying the model using a character vector or string scalar, not a formula or matrix.

Example: 'Intercept',false

Model specification describing terms that cannot be removed from the model, specified as the comma-separated pair consisting of 'Lower' and one of the options for modelspec naming the model.

Example: 'Lower','linear'

Offset variable in the fit, specified as the comma-separated pair consisting of 'Offset' and a vector or name of a variable with the same length as the response.

GeneralizedLinearModel.stepwise uses Offset as an additional predictor, with a coefficient value fixed at 1.0. In other words, the formula for fitting is

f(μ) ~ Offset + (terms involving real predictors)

with the Offset predictor having coefficient 1.

For example, consider a Poisson regression model. Suppose the number of counts is known for theoretical reasons to be proportional to a predictor A. By using the log link function and by specifying log(A) as an offset, you can force the model to satisfy this theoretical constraint.

Data Types: single | double | char | string

Threshold for the criterion to add a term, specified as the comma-separated pair consisting of 'PEnter' and a scalar value, as described in this table.

CriterionDefault ValueDecision
'Deviance'0.05If the p-value of F-statistic or chi-squared statistic is less than PEnter (p-value to enter), add the term to the model.
'SSE'0.05If the SSE of the model is less than PEnter, add the term to the model.
'AIC'0If the change in the AIC of the model is less than PEnter, add the term to the model.
'BIC'0If the change in the BIC of the model is less than PEnter, add the term to the model.
'Rsquared'0.1If the increase in the R-squared value of the model is greater than PEnter, add the term to the model.
'AdjRsquared'0If the increase in the adjusted R-squared value of the model is greater than PEnter, add the term to the model.

For more information, see the Criterion name-value pair argument.

Example: 'PEnter',0.075

Predictor variables to use in the fit, specified as the comma-separated pair consisting of 'PredictorVars' and either a string array or cell array of character vectors of the variable names in the table or dataset array tbl, or a logical or numeric index vector indicating which columns are predictor variables.

The string values or character vectors should be among the names in tbl, or the names you specify using the 'VarNames' name-value pair argument.

The default is all variables in X, or all variables in tbl except for ResponseVar.

For example, you can specify the second and third variables as the predictor variables using either of the following examples.

Example: 'PredictorVars',[2,3]

Example: 'PredictorVars',logical([0 1 1 0 0 0])

Data Types: single | double | logical | string | cell

Threshold for the criterion to remove a term, specified as the comma-separated pair consisting of 'PRemove' and a scalar value, as described in this table.

CriterionDefault ValueDecision
'Deviance'0.10If the p-value of F-statistic or chi-squared statistic is greater than PRemove (p-value to remove), remove the term from the model.
'SSE'0.10If the p-value of the F statistic is greater than PRemove, remove the term from the model.
'AIC'0.01If the change in the AIC of the model is greater than PRemove, remove the term from the model.
'BIC'0.01If the change in the BIC of the model is greater than PRemove, remove the term from the model.
'Rsquared'0.05If the increase in the R-squared value of the model is less than PRemove, remove the term from the model.
'AdjRsquared'-0.05If the increase in the adjusted R-squared value of the model is less than PRemove, remove the term from the model.

At each step, the GeneralizedLinearModel.stepwise function also checks whether a term is redundant (linearly dependent) with other terms in the current model. When any term is linearly dependent with other terms in the current model, the GeneralizedLinearModel.stepwise function removes the redundant term, regardless of the criterion value.

For more information, see the Criterion name-value pair argument.

Example: 'PRemove',0.05

Response variable to use in the fit, specified as the comma-separated pair consisting of 'ResponseVar' and either a character vector or string scalar containing the variable name in the table or dataset array tbl, or a logical or numeric index vector indicating which column is the response variable. You typically need to use 'ResponseVar' when fitting a table or dataset array tbl.

For example, you can specify the fourth variable, say yield, as the response out of six variables, in one of the following ways.

Example: 'ResponseVar','yield'

Example: 'ResponseVar',[4]

Example: 'ResponseVar',logical([0 0 0 1 0 0])

Data Types: single | double | logical | char | string

Model specification describing the largest set of terms in the fit, specified as the comma-separated pair consisting of 'Upper' and one of the options for modelspec naming the model.

Example: 'Upper','quadratic'

Names of variables, specified as the comma-separated pair consisting of 'VarNames' and a string array or cell array of character vectors including the names for the columns of X first, and the name for the response variable y last.

'VarNames' is not applicable to variables in a table or dataset array, because those variables already have names.

The variable names do not have to be valid MATLAB identifiers. However, if the names are not valid, you cannot use a formula when you fit or adjust a model; for example:

  • You cannot use a formula to specify the terms to add or remove when you use the addTerms function or the removeTerms function, respectively.

  • You cannot use a formula to specify the lower and upper bounds of the model when you use the step or stepwiseglm function with the name-value pair arguments 'Lower' and 'Upper', respectively.

Before specifying 'VarNames',varNames, you can verify the variable names in varNames by using the isvarname function. The following code returns logical 1 (true) for each variable that has a valid variable name.

cellfun(@isvarname,varNames)
If the variable names in varNames are not valid, then convert them by using the matlab.lang.makeValidName function.
varNames = matlab.lang.makeValidName(varNames);

Example: 'VarNames',{'Horsepower','Acceleration','Model_Year','MPG'}

Data Types: string | cell

Observation weights, specified as the comma-separated pair consisting of 'Weights' and an n-by-1 vector of nonnegative scalar values, where n is the number of observations.

Data Types: single | double

Output Arguments

expand all

Generalized linear model representing a least-squares fit of the link of the response to the data, returned as a GeneralizedLinearModel object.

For properties and methods of the generalized linear model object, mdl, see the GeneralizedLinearModel class page.

Examples

expand all

Create response data using just three of 20 predictors, and create a generalized linear model stepwise to see if it uses just the correct predictors.

Create data with 20 predictors, and Poisson response using just three of the predictors, plus a constant.

rng default % for reproducibility
X = randn(100,20);
mu = exp(X(:,[5 10 15])*[.4;.2;.3] + 1);
y = poissrnd(mu);

Fit a generalized linear model using the Poisson distribution.

mdl =  stepwiseglm(X,y,...
    'constant','upper','linear','Distribution','poisson')
1. Adding x5, Deviance = 134.439, Chi2Stat = 52.24814, PValue = 4.891229e-13
2. Adding x15, Deviance = 106.285, Chi2Stat = 28.15393, PValue = 1.1204e-07
3. Adding x10, Deviance = 95.0207, Chi2Stat = 11.2644, PValue = 0.000790094
mdl = 
Generalized linear regression model:
    log(y) ~ 1 + x5 + x10 + x15
    Distribution = Poisson

Estimated Coefficients:
                   Estimate       SE       tStat       pValue  
                   ________    ________    ______    __________

    (Intercept)     1.0115     0.064275    15.737    8.4217e-56
    x5             0.39508     0.066665    5.9263    3.0977e-09
    x10            0.18863      0.05534    3.4085     0.0006532
    x15            0.29295     0.053269    5.4995    3.8089e-08


100 observations, 96 error degrees of freedom
Dispersion: 1
Chi^2-statistic vs. constant model: 91.7, p-value = 9.61e-20

More About

expand all

Tips

  • The generalized linear model mdl is a standard linear model unless you specify otherwise with the Distribution name-value pair.

  • For other methods such as devianceTest, or properties of the GeneralizedLinearModel object, see GeneralizedLinearModel.

Algorithms

Stepwise regression is a systematic method for adding and removing terms from a linear or generalized linear model based on their statistical significance in explaining the response variable. The method begins with an initial model, specified using modelspec, and then compares the explanatory power of incrementally larger and smaller models.

The GeneralizedLinearModel.stepwise function uses forward and backward stepwise regression to determine a final model. At each step, the function searches for terms to add to the model or remove from the model based on the value of the 'Criterion' name-value pair argument.

The default value of 'Criterion' for a linear regression model is 'sse'. In this case, stepwiselm and step of LinearModel use the p-value of an F-statistic to test models with and without a potential term at each step. If a term is not currently in the model, the null hypothesis is that the term would have a zero coefficient if added to the model. If there is sufficient evidence to reject the null hypothesis, the function adds the term to the model. Conversely, if a term is currently in the model, the null hypothesis is that the term has a zero coefficient. If there is insufficient evidence to reject the null hypothesis, the function removes the term from the model.

Stepwise regression takes these steps when 'Criterion' is 'sse':

  1. Fit the initial model.

  2. Examine a set of available terms not in the model. If any of the terms have p-values less than an entrance tolerance (that is, if it is unlikely a term would have a zero coefficient if added to the model), add the term with the smallest p-value and repeat this step; otherwise, go to step 3.

  3. If any of the available terms in the model have p-values greater than an exit tolerance (that is, the hypothesis of a zero coefficient cannot be rejected), remove the term with the largest p-value and return to step 2; otherwise, end the process.

At any stage, the function will not add a higher-order term if the model does not also include all lower-order terms that are subsets of the higher-order term. For example, the function will not try to add the term X1:X2^2 unless both X1 and X2^2 are already in the model. Similarly, the function will not remove lower-order terms that are subsets of higher-order terms that remain in the model. For example, the function will not try to remove X1 or X2^2 if X1:X2^2 remains in the model.

The default value of 'Criterion' for a generalized linear model is 'Deviance'. stepwiseglm and step of GeneralizedLinearModel follow a similar procedure for adding or removing terms.

You can specify other criteria by using the 'Criterion' name-value pair argument. For example, you can specify the change in the value of the Akaike information criterion, Bayesian information criterion, R-squared, or adjusted R-squared as the criterion to add or remove terms.

Depending on the terms included in the initial model, and the order in which the function adds and removes terms, the function might build different models from the same set of potential terms. The function terminates when no single step improves the model. However, a different initial model or a different sequence of steps does not guarantee a better fit. In this sense, stepwise models are locally optimal, but might not be globally optimal.

Alternatives

You can also create a stepwise generalized linear model using stepwiseglm.

Use fitglm to create a model with a fixed specification. Use step, addTerms, or removeTerms to adjust a fitted model.

References

[1] Collett, D. Modeling Binary Data. New York: Chapman & Hall, 2002.

[2] Dobson, A. J. An Introduction to Generalized Linear Models. New York: Chapman & Hall, 1990.

[3] McCullagh, P., and J. A. Nelder. Generalized Linear Models. New York: Chapman & Hall, 1990.