Econometrics Toolbox™ supports modeling and analyzing discrete-time Markov models. These models describe stochastic processes that assume states xt in a state space X, subject to the Markov property, which requires the distribution of xt + 1 to be independent of the process history before reaching state xt.
Discrete state-space Markov processes, or Markov chains, are represented by a directed graph and described by a right-stochastic transition matrix P. The distribution of states at time t+1 is the distribution of states at time t multiplied by P. The structure of P determines the evolutionary trajectory of the chain, including asymptotics.
Continuous state-space Markov processes, or state-space models, allow for trajectories through a continuous state space. The underlying Markov process is typically unobserved. Supplemental observation equations describe the evolution of measurable characteristics of a system, dependent on the Markov process.