Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English verison of the page.

Note: This page has been translated by MathWorks. Please click here
To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

dtmc

Create discrete-time Markov chain

Description

dtmc creates a discrete-time, finite-state, time-homogeneous Markov chain from a specified state transition matrix.

After creating a dtmc object, you can analyze the structure and evolution of the Markov chain, or visualize the Markov chain various ways, using the object functions.

Creation

Syntax

mc = dtmc(P)
mc = dtmc(P,'StateNames',stateNames)

Description

example

mc = dtmc(P) creates a discrete-time Markov chain object mc specified by the state transition matrix P.

example

mc = dtmc(P,'StateNames',stateNames) optionally associates names stateNames to the states.

Input Arguments

expand all

State transition matrix, specified as a numStates-by-numStates nonnegative numeric matrix.

P(i,j) is either the theoretical probability of a transition from state i to state j or an empirical count of observed transitions from state i to state j.

dtmc normalizes each row of P to sum to 1, then stores the normalized matrix in the property P.

Data Types: double

Properties

expand all

This property is read-only.

Normalized transition matrix, specified as a numStates-by-numStates nonnegative numeric matrix.

If x is a row vector of length numStates specifying a distribution of states at time t (x sums to one), then x*P is the distribution of states at time t + 1.

Data Types: double

This property is read-only.

Number of states, specified as a positive scalar.

Data Types: double

State labels, specified as a string vector, cell vector of character vectors, or numeric vector of length numStates. Elements correspond to rows and columns of P.

Example: ["Depression" "Recession" "Stagnant" "Boom"]

Data Types: string

Object Functions

asymptoticsDetermine Markov chain asymptotics
isergodicCheck Markov chain for ergodicity
isreducibleCheck Markov chain for reducibility
classifyClassify Markov chain states
lazyAdjust Markov chain state inertia
subchainExtract Markov subchain
redistributeCompute Markov chain redistributions
simulateSimulate Markov chain state walks
distplotPlot Markov chain redistributions
eigplotPlot Markov chain eigenvalues
graphplotPlot Markov chain directed graph
simplotPlot Markov chain simulations

Examples

expand all

Consider this theoretical, right-stochastic transition matrix of a stochastic process.

Element is the probability that the process transitions to state j at time t + 1 given that it is in state i at time t, for all t.

Create the Markov chain that is characterized by the transition matrix P.

P = [0.5 0.5 0 0; 0.5 0 0.5 0; 0 0 0 1; 0 0 1 0];
mc = dtmc(P);

mc is a dtmc object that represents the Markov chain.

Display the number of states in the Markov chain.

numstates = mc.NumStates
numstates =

     4

Plot a directed graph of the Markov chain.

figure;
graphplot(mc);

Observe that states 3 and 4 form an absorbing class, while states 1 and 2 are transient.

Consider this transition matrix in which element is the observed number of times state i transitioned to state j.

For example, implies state 3 transitioned to state 2 seven times.

P = [16 2  3  13;
     5  11 10 8;
     9  7  6  12;
     4  14 15 1];

Create the Markov chain that is characterized by the transition matrix P.

mc = dtmc(P);

Display the normalized transition matrix stored in MC. Verify that the elements within rows sum to 1 for all rows.

mc.P
sum(mc.P,2)
ans =

    0.4706    0.0588    0.0882    0.3824
    0.1471    0.3235    0.2941    0.2353
    0.2647    0.2059    0.1765    0.3529
    0.1176    0.4118    0.4412    0.0294


ans =

     1
     1
     1
     1

Plot a directed graph of the Markov chain.

figure;
graphplot(mc);

Consider the two-state business cycle of the U.S. real gross national product (GNP) in [3] p. 697. At time t, real GNP can be in a state of expansion or contraction. Suppose that the following statements are true during the sample period.

  • If real GNP is expanding at time t, then the probability that it will continue in an expansion state at time t + 1 is .

  • If real GNP is contracting at time t, then the probability that it will continue in a contraction state at time t + 1 is .

Create the transition matrix for the model.

p11 = 0.90;
p22 = 0.75;

P = [p11 (1 - p11); (1 - p22) p22];

Create the Markov chain that is characterized by the transition matrix P. Label the two states.

mc = dtmc(P,'StateNames',["Expansion" "Contraction"])
mc = 

  dtmc with properties:

             P: [2x2 double]
    StateNames: ["Expansion"    "Contraction"]
     NumStates: 2

Plot a directed graph of the Markov chain. Indicate the probability of transition using edge colors.

figure;
graphplot(mc,'ColorEdges',true);

To help you explore the dtmc object functions, mcmix creates a Markov Chain from a random transition matrix using only a specified number of states.

Create a five-state Markov chain from a random transition matrix.

rng(1); % For reproducibility
mc = mcmix(5)
mc = 

  dtmc with properties:

             P: [5x5 double]
    StateNames: ["1"    "2"    "3"    "4"    "5"]
     NumStates: 5

MC is a dtmc object.

Plot the eigenvalues of the transition matrix on the complex plane.

figure;
eigplot(mc)

This spectrum determines structural properties of the Markov chain, such as periodicity and mixing rate.

Alternatives

You also can create a Markov chain object using mcmix.

References

[1] Gallager, R.G. Stochastic Processes: Theory for Applications. Cambridge, UK: Cambridge University Press, 2013.

[2] Haggstrom, O. Finite Markov Chains and Algorithmic Applications. Cambridge, UK: Cambridge University Press, 2002.

[3] Hamilton, J. D. Time Series Analysis. Princeton, NJ: Princeton University Press, 1994.

[4] Norris, J. R. Markov Chains. Cambridge, UK: Cambridge University Press, 1997.

Introduced in R2017b

Was this topic helpful?