## Maximum Likelihood Estimation for Conditional Mean Models

### Innovation Distribution

For conditional mean models in Econometrics Toolbox™, the form of the innovation process is${\epsilon }_{t}={\sigma }_{t}{z}_{t},$ where zt can be standardized Gaussian or Student’s t with $\nu >2$ degrees of freedom. Specify your distribution choice in the `arima` model object `Distribution` property.

The innovation variance, ${\sigma }_{t}^{2},$ can be a positive scalar constant, or characterized by a conditional variance model. Specify the form of the conditional variance using the `Variance` property. If you specify a conditional variance model, the parameters of that model are estimated with the conditional mean model parameters simultaneously.

Given a stationary model,

`${y}_{t}=\mu +\psi \left(L\right){\epsilon }_{t},$`

applying an inverse filter yields a solution for the innovation ${\epsilon }_{t}$

`${\epsilon }_{t}={\psi }^{-1}\left(L\right)\left({y}_{t}-\mu \right).$`

For example, for an AR(p) process,

`${\epsilon }_{t}=-c+\varphi \left(L\right){y}_{t},$`

where $\varphi \left(L\right)=\left(1-{\varphi }_{1}L-\cdots -{\varphi }_{p}{L}^{p}\right)$ is the degree p AR operator polynomial.

`estimate` uses maximum likelihood to estimate the parameters of an `arima` model. `estimate` returns fitted values for any parameters in the input model object equal to `NaN`. `estimate` honors any equality constraints in the input model object, and does not return estimates for parameters with equality constraints.

### Loglikelihood Functions

Given the history of a process, innovations are conditionally independent. Let Ht denote the history of a process available at time t, t = 1,...,N. The likelihood function for the innovation series is given by

`$f\left({\epsilon }_{1},{\epsilon }_{2},\dots ,{\epsilon }_{N}|{H}_{N-1}\right)=\prod _{t=1}^{N}f\left({\epsilon }_{t}|{H}_{t-1}\right),$`

where f is a standardized Gaussian or t density function.

The exact form of the loglikelihood objective function depends on the parametric form of the innovation distribution.

• If zt has a standard Gaussian distribution, then the loglikelihood function is

`$LLF=-\frac{N}{2}\mathrm{log}\left(2\pi \right)-\frac{1}{2}\sum _{t=1}^{N}\mathrm{log}{\sigma }_{t}^{2}-\frac{1}{2}\sum _{t=1}^{N}\frac{{\epsilon }_{t}^{2}}{{\sigma }_{t}^{2}}.$`

• If zt has a standardized Student’s t distribution with $\nu >2$ degrees of freedom, then the loglikelihood function is

`$LLF=N\mathrm{log}\left[\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\sqrt{\pi \left(\nu -2\right)}\Gamma \left(\frac{\nu }{2}\right)}\right]-\frac{1}{2}\sum _{t=1}^{N}\mathrm{log}{\sigma }_{t}^{2}-\frac{\nu +1}{2}\sum _{t=1}^{N}\mathrm{log}\left[1+\frac{{\epsilon }_{t}^{2}}{{\sigma }_{t}^{2}\left(\nu -2\right)}\right].$`

`estimate` performs covariance matrix estimation for maximum likelihood estimates using the outer product of gradients (OPG) method.