Consider this theoretical, right-stochastic transition matrix of a stochastic process.
Create the Markov chain that is characterized by the transition matrix P.
Plot a digraph of the Markov chain mc
. Display the transition probabilities.
Compute the expected first hitting time for state 1, beginning from each state in the Markov chain.
ht = 4×1
0
2.3333
4.0000
3.6667
Plot a digraph of the Markov chain. Specify node colors representing the expected first hitting times for state 1, beginning from each state in the Markov chain.
Plot another digraph. Include state 4 as a target state.
Create the Markov chain characterized by this transition matrix:
Compute the expected first hitting times for state 1, beginning from each state in the Markov chain mc
. Also, plot a digraph and specify node colors representing the expected first hitting times for state 1.
ht = 7×1
0
Inf
4
Inf
Inf
Inf
2
States 2 and 4 form an absorbing class. Therefore, state 1 is unreachable from these states. The absorbing class is remote with respect to state 1, with an expected first hitting time of Inf
.
State 1 is reachable from states 5 and 6, but the probability of transitioning into the absorbing class from states 5 and 6 is nonzero. Therefore, states 5 and 6 are remote-reachable with respect to state 1, with an expected first hitting time of Inf
.
The expected first hitting time for state 1 beginning from state 7 is 2 time steps. The expected first hitting time for state 1 beginning from state 3 is 4 time steps.