Note: This page has been translated by MathWorks. Please click here

To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

Linear mixed-effects models are extensions of linear regression models for data that are collected and summarized in groups. These models describe the relationship between a response variable and independent variables, with coefficients that can vary with respect to one or more grouping variables. A mixed-effects model consists of two parts, fixed effects and random effects. Fixed-effects terms are usually the conventional linear regression part, and the random effects are associated with individual experimental units drawn at random from a population. The random effects have prior distributions whereas fixed effects do not. Mixed-effects models can represent the covariance structure related to the grouping of data by associating the common random effects to observations that have the same level of a grouping variable. The standard form of a linear mixed-effects model is

$$y=\underset{fixed}{\underbrace{X\beta}}+\underset{random}{\underbrace{Zb}}+\underset{error}{\underbrace{\epsilon}},$$

where

*y*is the*n*-by-1 response vector, and*n*is the number of observations.*X*is an*n*-by-*p*fixed-effects design matrix.*β*is a*p*-by-1 fixed-effects vector.*Z*is an*n*-by-*q*random-effects design matrix.*b*is a*q*-by-1 random-effects vector.*ε*is the*n*-by-1 observation error vector.

The assumptions for the linear mixed-effects model are:

Random-effects vector,

*b*, and the error vector,*ε*, have the following prior distributions:$$\begin{array}{l}b~N\left(0,{\sigma}^{2}D\left(\theta \right)\right),\\ \epsilon ~N\left(0,\sigma {}^{2}I\right),\end{array}$$

where

*D*is a symmetric and positive semidefinite matrix, parameterized by a variance component vector*θ*,*I*is an*n*-by-*n*identity matrix, and*σ*^{2}is the error variance.Random-effects vector,

*b*, and the error vector,*ε*, are independent from each other.

Mixed-effects models are also called *multilevel models* or *hierarchical
models* depending on the context. Mixed-effects models is
a more general term than the latter two. Mixed-effects models might
include factors that are not necessarily multilevel or hierarchical,
for example crossed factors. That is why mixed-effects is the terminology
preferred here. Sometimes mixed-effects models are expressed as multilevel
regression models (first level and grouping level models) that are
fit simultaneously. For example, a varying or random intercept model,
with one continuous predictor variable *x* and one
grouping variable with *M* levels, can be expressed
as

$$\begin{array}{l}{y}_{im}={\beta}_{0m}+{\beta}_{1}{x}_{im}+{\epsilon}_{im},\text{\hspace{1em}}i=1,2,\mathrm{..},n,\text{\hspace{1em}}m=1,2,\mathrm{...},M,\text{\hspace{1em}}{\epsilon}_{im}~N\left(0,{\sigma}^{2}\right),\\ {\beta}_{0m}={\beta}_{00}+{b}_{0m},\text{\hspace{1em}}{b}_{0m}~N\left(0,{\sigma}_{0}^{2}\right),\end{array}$$

where *y*_{im} corresponds
to data for observation *i* and group *m*, *n* is
the total number of observations, and b_{0m} and
ε_{im} are independent
of each other. After substituting the group-level parameters in the
first-level model, the model for the response vector becomes

$${y}_{im}=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta}_{00}+{\beta}_{1}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}}}+{\epsilon}_{im}.$$

A random intercept and slope model with one continuous
predictor variable *x*, where both the intercept
and slope vary independently by a grouping variable with *M* levels
is

$$\begin{array}{l}{y}_{im}={\beta}_{0m}+{\beta}_{1m}{x}_{im}+{\epsilon}_{im},\text{\hspace{1em}}i=1,2,\mathrm{...},n,\text{\hspace{1em}}m=1,2,\mathrm{...},M,\text{\hspace{1em}}{\epsilon}_{im}~N\left(0,{\sigma}^{2}\right),\\ {\beta}_{0m}={\beta}_{00}+{b}_{0m},\text{\hspace{1em}}{b}_{0m}~N\left(0,{\sigma}_{0}^{2}\right),\text{\hspace{1em}}\\ {\beta}_{1m}={\beta}_{10}+{b}_{1m},\text{\hspace{1em}}{b}_{1m}~N\left(0,{\sigma}_{1}^{2}\right),\end{array}$$

or

$${b}_{m}=\left(\begin{array}{l}{b}_{0m}\\ {b}_{1m}\end{array}\right)~N\left(0,\left(\begin{array}{cc}{\sigma}_{0}^{2}& 0\\ 0& {\sigma}_{1}^{2}\end{array}\right)\right).$$

You might also have correlated random effects. In general, for a model with a random intercept and slope, the distribution of the random effects is

$${b}_{m}=\left(\begin{array}{l}{b}_{0m}\\ {b}_{1m}\end{array}\right)~N\left(0,\sigma {}^{2}D\left(\theta \right)\right),$$

where *D* is a 2-by-2 symmetric
and positive semidefinite matrix, parameterized by a variance component
vector *θ*.

After substituting the group-level parameters in the first-level model, the model for the response vector is

$${y}_{im}=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta}_{00}+{\beta}_{10}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}+{b}_{1m}{x}_{im}}}+{\epsilon}_{im},\text{\hspace{1em}}i=1,2,\mathrm{...},n,\text{\hspace{1em}}m=1,2,\mathrm{...},M.$$

If you express the group-level variable, *x*_{im},
in the random-effects term by *z*_{im},
this model is

$${y}_{im}=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta}_{00}+{\beta}_{10}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}+{b}_{1m}{z}_{im}}}+{\epsilon}_{im},\text{\hspace{1em}}i=1,2,\mathrm{...},n,\text{\hspace{1em}}m=1,2,\mathrm{...},M.$$

In this case, the same terms appear in both the fixed-effects
design matrix and random-effects design matrix. Each *z _{im}* and

It is also possible to explain more of the group-level variations
by adding more group-level predictor variables. A random-intercept
and random-slope model with one continuous predictor variable *x*,
where both the intercept and slope vary independently by a grouping
variable with *M* levels, and one group-level predictor
variable *v*_{m} is

$$\begin{array}{l}{y}_{im}={\beta}_{0im}+{\beta}_{1im}{x}_{im}+{\epsilon}_{im},\text{\hspace{1em}}i=1,2,\mathrm{...},n,\text{\hspace{1em}}m=1,2,\mathrm{...},M,\text{\hspace{1em}}{\epsilon}_{im}~N\left(0,{\sigma}^{2}\right),\\ {\beta}_{0im}={\beta}_{00}+{\beta}_{01}{v}_{im}+{b}_{0m},\text{\hspace{1em}}{b}_{0m}~N\left(0,{\sigma}_{0}^{2}\right),\text{\hspace{1em}}\\ {\beta}_{1im}={\beta}_{10}+{\beta}_{11}{v}_{im}+{b}_{1m},\text{\hspace{1em}}{b}_{1m}~N\left(0,{\sigma}_{1}^{2}\right).\end{array}$$

This model results in main effects of the group-level predictor and an interaction term between the first-level and group-level predictor variables in the model for the response variable as

$$\begin{array}{l}{y}_{im}={\beta}_{00}+{\beta}_{01}{v}_{im}+{b}_{0m}+\left({\beta}_{10}+{\beta}_{11}{v}_{im}+{b}_{1m}\right){x}_{im}+{\epsilon}_{im},\text{\hspace{1em}}i=1,2,\mathrm{...},n,\text{\hspace{1em}}m=1,2,\mathrm{...},M,\\ \text{\hspace{1em}}\text{\hspace{1em}}=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta}_{00}+{\beta}_{10}{x}_{im}+{\beta}_{01}{v}_{im}+{\beta}_{11}{v}_{im}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}+{b}_{1m}{x}_{im}}}+{\epsilon}_{im}.\end{array}$$

The term *β*_{11}*v*_{m}*x*_{im} is
often called a cross-level interaction in many textbooks on multilevel
models. The model for the response variable *y* can
be expressed as

$$\begin{array}{l}{y}_{im}=\left[\begin{array}{cccc}1& {x}_{1}{}_{im}& {v}_{im}& {v}_{im}{x}_{1im}\end{array}\right]\left[\begin{array}{c}{\beta}_{00}\\ {\beta}_{10}\\ {\beta}_{01}\\ {\beta}_{11}\end{array}\right]+\left[\begin{array}{cc}1& {x}_{1im}\end{array}\right]\left[\begin{array}{c}{b}_{0m}\\ {b}_{1m}\end{array}\right]+{\epsilon}_{im},\text{\hspace{1em}}i=1,2,\mathrm{...},n,\text{\hspace{1em}}m=1,2,\mathrm{...},M,\\ \text{\hspace{1em}}\text{\hspace{1em}}\end{array}$$

which corresponds to the standard form given earlier,

$$\text{\hspace{1em}}y=X\beta +Zb+\epsilon .$$

In general, if there are *R* grouping variables,
and *m*(*r*,*i*)
shows the level of grouping variable *r*, for observation *i*,
then the model for the response variable for observation *i* is

$${y}_{i}={x}_{i}^{T}\beta +{\displaystyle \sum _{r=1}^{R}{z}_{ir}{b}_{m(r,i)}^{(r)}}+{\epsilon}_{i},\text{\hspace{1em}}i=1,2,\mathrm{...},n,$$

where *β* is a *p*-by-1
fixed-effects vector, *b*^{(r)}_{m(r,i)} is
a *q*(*r*)-by-1 random-effects vector
for the* r*th grouping variable and level *m*(*r*,*i*),
and *ε*_{i} is
a 1-by-1 error term for observation *i*.

[1] Pinherio, J. C., and D. M. Bates. *Mixed-Effects
Models in S and S-PLUS*. Statistics and Computing Series,
Springer, 2004.

[2] Hariharan, S. and J. H. Rogers. "Estimation Procedures
for Hierarchical Linear Models." *Multilevel Modeling
of Educational Data* (A. A. Connell and D. B. McCoach,
eds.). Charlotte, NC: Information Age Publishing, Inc., 2008.

[3] Hox, J. *Multilevel Analysis, Techniques and
Applications*. Lawrence Erlbaum Associates, Inc., 2002

[4] Snidjers, T. and R. Bosker. *Multilevel Analysis*.
Thousand Oaks, CA: Sage Publications, 1999.

[5] Gelman, A. and J. Hill. *Data Analysis Using
Regression and Multilevel/Hierarchical Models*. New York,
NY: Cambridge University Press, 2007.

`fitlme`

| `fitlmematrix`

| `LinearMixedModel`

Was this topic helpful?