Econometrics Toolbox™ supports modeling and analysis of discrete-time Markov models. These models describe stochastic processes that assume states xt in a state space X, subject to the Markov property, which requires the distribution of xt + 1 to be independent of the history of the process before reaching state xt.
Discrete-state Markov processes, or Markov chains, are represented by a directed graph and described by a right-stochastic transition matrix P. The distribution of states at time t+1 is the distribution of states at time t, multiplied by P. The structure of P determines the evolutionary trajectory of the chain, including asymptotics.
Continuous-state Markov processes, or state-space models, allow for trajectories through a continuous state space. The underlying Markov process is typically unobserved. Supplemental observation equations describe the evolution of measurable characteristics of a system, dependent on the Markov process.