When you fit multivariate linear regression models using
you can use the optional name-value pair
choose least squares estimation. In this case, by default,
ordinary least squares (OLS) estimates using . Alternatively, if you specify
a covariance matrix for weighting, you can return covariance-weighted
least squares (CWLS) estimates. If you combine OLS and CWLS, you can
get feasible generalized least squares (FGLS) estimates.
The OLS estimate for the coefficient vector is the vector that minimizes
default), the variance-covariance matrix of the OLS estimates is
mvregressoutput. The standard errors of the OLS regression coefficients are the square root of the diagonal of this variance-covariance matrix.
If your data is not scaled such that , then you can multiply the
matrix by the mean squared error (MSE), an unbiased estimate of . To compute the MSE, return
the n-by-d matrix of residuals, (the third
For most multivariate problems, an identity error covariance
matrix is insufficient, and leads to inefficient or biased standard
error estimates. You can specify a matrix for CWLS estimation using
the optional name-value pair argument
example, an invertible d-by-d matrix
named . Usually, is a diagonal matrix such that
the inverse matrix contains weights
for each dimension to model heteroscedasticity. However, can also be a nondiagonal matrix
that models correlation.
Given , the CWLS solution is the vector that minimizes
If , this is the generalized least squares (GLS) solution. The corresponding variance-covariance matrix of the CWLS estimates is
mvregressoutput. The standard errors of the CWLS regression coefficients are the square root of the diagonal of this variance-covariance matrix.
If you only know the error covariance matrix up to a proportion,
that is, , you can multiply the
matrix by the MSE, as described in Ordinary Least Squares.
Regardless of which least squares method you use, the estimate for the error variance-covariance matrix is
The error covariance estimate, ,
is the second
mvregress output, and the matrix
of residuals, , is the third
output. If you specify the optional name-value pair
mvregress returns with zeros in the off-diagonal
The generalized least squares estimate is the CWLS estimate with a known covariance matrix. That is, given is known, the GLS solution is
In most cases, the error covariance is unknown. The feasible generalized least squares (FGLS) estimate uses in place of . You can obtain two-step FGLS estimates as follows:
Perform OLS regression, and return an estimate .
Perform CWLS regression, using .
You can also iterate between these two steps until convergence is reached.
For some data, the OLS estimate is positive semidefinite, and
has no unique inverse. In this case, you cannot get the FGLS estimate
mvregress. As an alternative, you can use
which uses a generalized inverse to return weighted least squares
solutions for positive semidefinite covariance matrices.
An alternative to FGLS is to use OLS coefficient estimates (which are consistent) and make a standard error correction to improve efficiency. One such standard error adjustment—which does not require inversion of the covariance matrix—is panel corrected standard errors (PCSE) . The panel corrected variance-covariance matrix for OLS estimates is
The default estimation algorithm used by
maximum likelihood estimation (MLE). The loglikelihood function for
the multivariate linear regression model is
mvregress finds the MLEs using an iterative
two-stage algorithm. At iteration m + 1, the estimates
The algorithm terminates when the changes in the coefficient
estimates and loglikelihood objective function are less than a specified
tolerance, or when the specified maximum number of iterations is reached.
The optional name-value pair arguments for changing these convergence
The variance-covariance matrix of the MLEs is an optional
mvregress returns the variance-covariance
matrix for only the regression coefficients, but you can also get
the variance-covariance matrix of using
the optional name-value pair
In this case,
mvregress returns the variance-covariance
matrix for all K regression coefficients, and d or d(d +
1)/2 covariance terms (depending on whether the error covariance is
diagonal or full).
By default, the variance-covariance matrix is the inverse of
the observed Fisher information matrix (the
You can request the expected Fisher information matrix using the optional
'vartype','fisher'. Provided there
is no missing response data, the observed and expected Fisher information
matrices are the same. If response data is missing, the observed Fisher
information accounts for the added uncertainty due to the missing
values, whereas the expected Fisher information matrix does not.
The variance-covariance matrix for the regression coefficient MLEs is
mvregressoutput. The standard errors of the MLEs are the square root of the diagonal of this variance-covariance matrix.
For , let denote the vector of parameters in the estimated error variance-covariance matrix. For example, if d = 2, then:
If the estimated covariance matrix is diagonal, then .
If the estimated covariance matrix is full, then .
The Fisher information matrix for , , has elements
When you request the full variance-covariance matrix,
(as the fourth output) the block diagonal matrix
If any response values are missing, indicated by
an expectation/conditional maximization (ECM) algorithm for estimation
(if enough data is available). In this case, the algorithm is iterative
for both least squares and maximum likelihood estimation. During each
mvregress imputes missing response values
using their conditional expectation.
Consider organizing the data so that the joint distribution of the missing and observed responses, denoted and respectively, can be written as
mvregressuses the parameter values from the previous iteration to:
Update the regression coefficients using the combined vector of observed responses and conditional expectations of missing responses.
Update the variance-covariance matrix, adjusting for missing responses using the variance-covariance matrix of the conditional distribution.
Finally, the residuals that
for missing responses are the difference between the conditional expectation
and the fitted value, both evaluated at the final parameter estimates.
If you prefer to ignore any observations that have missing response
values, use the name-value pair
mvregress always ignores observations
that have missing predictor values.
mvregress uses the observed Fisher
information matrix (the
'hessian' option) to compute
the variance-covariance matrix of the regression parameters. This
accounts for the additional uncertainty due to missing response values.
The observed information matrix includes contributions from only the observed responses. That is, the observed Fisher information matrix for the parameters in the error variance-covariance matrix has elements
For example, if d = 3, but is missing, then
 Beck, N. and J. N. Katz. What to Do (and Not to Do) with Time-Series-Cross-Section Data in Comparative Politics. American Political Science Review, Vol. 89, No. 3, pp. 634–647, 1995.