Class: GeneralizedLinearModel
Create generalized linear regression model by stepwise regression
GeneralizedLinearModel.stepwise
will be removed
in a future release. Use stepwiseglm
instead.
mdl = GeneralizedLinearModel.stepwise(tbl,modelspec)
mdl = GeneralizedLinearModel.stepwise(X,y,modelspec)
mdl = GeneralizedLinearModel.stepwise(...,modelspec,Name,Value)
creates
a generalized linear model of a table or dataset array mdl
= GeneralizedLinearModel.stepwise(tbl
,modelspec
)tbl
,
using stepwise regression to add or remove predictors. modelspec
is
the starting model for the stepwise procedure.
creates
a generalized linear model of the responses mdl
= GeneralizedLinearModel.stepwise(X
,y
,modelspec
)y
to
a data matrix X
, using stepwise regression to add
or remove predictors.
creates
a generalized linear model with additional options specified by one
or more mdl
= GeneralizedLinearModel.stepwise(...,modelspec
,Name,Value
)Name,Value
pair arguments.
The generalized linear model mdl
is
a standard linear model unless you specify otherwise with the Distribution
namevalue
pair.
For other methods such as devianceTest
,
or properties of the GeneralizedLinearModel
object,
see GeneralizedLinearModel
.
A terms matrix is a tby(p + 1) matrix specifying terms in a model, where t is the number of terms, p is the number of predictor variables, and plus one is for the response variable.
The value of T(i,j)
is the exponent of variable j
in
term i
. Suppose there are three predictor variables A
, B
,
and C
:
[0 0 0 0] % Constant term or intercept [0 1 0 0] % B; equivalently, A^0 * B^1 * C^0 [1 0 1 0] % A*C [2 0 0 0] % A^2 [0 1 2 0] % B*(C^2)
0
at the
end of each term represents the response variable. In general,If you have the variables in a table or dataset array,
then 0
must represent the response variable depending
on the position of the response variable. The following example illustrates
this.
Load the sample data and define the dataset array.
load hospital ds = dataset(hospital.Sex,hospital.BloodPressure(:,1),hospital.Age,... hospital.Smoker,'VarNames',{'Sex','BloodPressure','Age','Smoker'});
Represent the linear model 'BloodPressure ~ 1 + Sex
+ Age + Smoker'
in a terms matrix. The response variable
is in the second column of the dataset array, so there must be a column
of 0s for the response variable in the second column of the terms
matrix.
T = [0 0 0 0;1 0 0 0;0 0 1 0;0 0 0 1]
T = 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1
Redefine the dataset array.
ds = dataset(hospital.BloodPressure(:,1),hospital.Sex,hospital.Age,... hospital.Smoker,'VarNames',{'BloodPressure','Sex','Age','Smoker'});
Now, the response variable is the first term in the dataset
array. Specify the same linear model, 'BloodPressure ~ 1
+ Sex + Age + Smoker'
, using a terms matrix.
T = [0 0 0 0;0 1 0 0;0 0 1 0;0 0 0 1]
T = 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1
If you have the predictor and response variables in
a matrix and column vector, then you must include 0
for
the response variable at the end of each term. The following example
illustrates this.
Load the sample data and define the matrix of predictors.
load carsmall
X = [Acceleration,Weight];
Specify the model 'MPG ~ Acceleration + Weight + Acceleration:Weight
+ Weight^2'
using a term matrix and fit the model to the
data. This model includes the main effect and twoway interaction
terms for the variables, Acceleration
and Weight
,
and a secondorder term for the variable, Weight
.
T = [0 0 0;1 0 0;0 1 0;1 1 0;0 2 0]
T = 0 0 0 1 0 0 0 1 0 1 1 0 0 2 0
Fit a linear model.
mdl = fitlm(X,MPG,T)
mdl = Linear regression model: y ~ 1 + x1*x2 + x2^2 Estimated Coefficients: Estimate SE tStat pValue (Intercept) 48.906 12.589 3.8847 0.00019665 x1 0.54418 0.57125 0.95261 0.34337 x2 0.012781 0.0060312 2.1192 0.036857 x1:x2 0.00010892 0.00017925 0.6076 0.545 x2^2 9.7518e07 7.5389e07 1.2935 0.19917 Number of observations: 94, Error degrees of freedom: 89 Root Mean Squared Error: 4.1 Rsquared: 0.751, Adjusted RSquared 0.739 Fstatistic vs. constant model: 67, pvalue = 4.99e26
Only the intercept and x2
term, which correspond
to the Weight
variable, are significant at the
5% significance level.
Now, perform a stepwise regression with a constant model as the starting model and a linear model with interactions as the upper model.
T = [0 0 0;1 0 0;0 1 0;1 1 0];
mdl = stepwiselm(X,MPG,[0 0 0],'upper',T)
1. Adding x2, FStat = 259.3087, pValue = 1.643351e28 mdl = Linear regression model: y ~ 1 + x2 Estimated Coefficients: Estimate SE tStat pValue (Intercept) 49.238 1.6411 30.002 2.7015e49 x2 0.0086119 0.0005348 16.103 1.6434e28 Number of observations: 94, Error degrees of freedom: 92 Root Mean Squared Error: 4.13 Rsquared: 0.738, Adjusted RSquared 0.735 Fstatistic vs. constant model: 259, pvalue = 1.64e28
The results of the stepwise regression are consistent with the
results of fitlm
in the previous step.
A formula for model specification is a string
of the form '
Y
~ terms
'
where
Y
is the response name.
terms
contains
Variable names
+
means include the next variable

means do not include the next
variable
:
defines an interaction, a product
of terms
*
defines an interaction and all lowerorder terms
^
raises the predictor to a power,
exactly as in *
repeated, so ^
includes
lower order terms as well
()
groups terms
Note:
Formulas include a constant (intercept) term by default. To
exclude a constant term from the model, include 
For example,
'Y ~ A + B + C'
means a threevariable
linear model with intercept.'Y ~ A + B +
C  1'
is a threevariable linear model without intercept.'Y ~ A + B + C + B^2'
is a threevariable
model with intercept and a B^2
term.'Y
~ A + B^2 + C'
is the same as the previous example because B^2
includes
a B
term.'Y ~ A + B +
C + A:B'
includes an A*B
term.'Y
~ A*B + C'
is the same as the previous example because A*B
= A + B + A:B
.'Y ~ A*B*C  A:B:C'
has
all interactions among A
, B
,
and C
, except the threeway interaction.'Y
~ A*(B + C + D)'
has all linear terms, plus products of A
with
each of the other variables.
Wilkinson notation describes the factors present in models. The notation relates to factors present in models, not to the multipliers (coefficients) of those factors.
Wilkinson Notation  Factors in Standard Notation 

1  Constant (intercept) term 
A^k , where k is a positive
integer  A , A^{2} ,
..., A^{k} 
A + B  A , B 
A*B  A , B , A*B 
A:B  A*B only 
B  Do not include B 
A*B + C  A , B , C , A*B 
A + B + C + A:B  A , B , C , A*B 
A*B*C  A:B:C  A , B , C , A*B , A*C , B*C 
A*(B + C)  A , B , C , A*B , A*C 
Statistics and Machine Learning Toolbox™ notation always includes a constant term
unless you explicitly remove the term using 1
.
The default link function for a generalized linear model is the canonical link function.
Canonical Link Functions for Generalized Linear Models
Distribution  Link Function Name  Link Function  Mean (Inverse) Function 

'normal'  'identity'  f(μ) = μ  μ = Xb 
'binomial'  'logit'  f(μ) = log(μ/(1–μ))  μ = exp(Xb) / (1 + exp(Xb)) 
'poisson'  'log'  f(μ) = log(μ)  μ = exp(Xb) 
'gamma'  1  f(μ) = 1/μ  μ = 1/(Xb) 
'inverse gaussian'  2  f(μ) = 1/μ^{2}  μ = (Xb)^{–1/2} 
Stepwise regression is a systematic method
for adding and removing terms from a linear or generalized linear
model based on their statistical significance in explaining the response
variable. The method begins with an initial model, specified using modelspec
,
and then compares the explanatory power of incrementally larger and
smaller models.
MATLAB^{®} uses forward and backward stepwise regression to
determine a final model. At each step, the method searches for terms
to add to or remove from the model based on the value of the 'Criterion'
argument.
The default value of 'Criterion'
is 'sse'
,
and in this case, stepwiselm
uses the pvalue
of an Fstatistic to test models with and without
a potential term at each step. If a term is not currently in the model,
the null hypothesis is that the term would have a zero coefficient
if added to the model. If there is sufficient evidence to reject the
null hypothesis, the term is added to the model. Conversely, if a
term is currently in the model, the null hypothesis is that the term
has a zero coefficient. If there is insufficient evidence to reject
the null hypothesis, the term is removed from the model.
Here is how stepwise proceeds when 'Criterion'
is 'sse'
:
Fit the initial model.
If any terms not in the model have pvalues less than an entrance tolerance (that is, if it is unlikely that they would have zero coefficient if added to the model), add the one with the smallest pvalue and repeat this step; otherwise, go to step 3.
If any terms in the model have pvalues greater than an exit tolerance (that is, the hypothesis of a zero coefficient can be rejected), remove the one with the largest pvalue and go to step 2; otherwise, end.
The default for stepwiseglm
is 'Deviance'
and
it follows a similar procedure for adding or removing terms.
There are several other criteria available, which you can specify
using the 'Criterion'
argument. You can use the
change in the value of the Akaike information criterion, Bayesian
information criterion, Rsquared, adjusted Rsquared as a criterion
to add or remove terms.
Depending on the terms included in the initial model and the order in which terms are moved in and out, the method might build different models from the same set of potential terms. The method terminates when no single step improves the model. There is no guarantee, however, that a different initial model or a different sequence of steps will not lead to a better fit. In this sense, stepwise models are locally optimal, but might not be globally optimal.
You can also create a stepwise generalized linear model using stepwiseglm
.
Use fitglm
to create a
model with a fixed specification. Use step
, addTerms
,
or removeTerms
to adjust a fitted model.
[1] Collett, D. Modeling Binary Data. New York: Chapman & Hall, 2002.
[2] Dobson, A. J. An Introduction to Generalized Linear Models. New York: Chapman & Hall, 1990.
[3] McCullagh, P., and J. A. Nelder. Generalized Linear Models. New York: Chapman & Hall, 1990.