3

We have a markov chain with states $\{1,2,3,4,5 \}$ with $A = \{1,5\}$ being our absorbing states. We take a step forward with probability P, and step backwards with probability Q.

Let $T_k $ be the number of steps to hit $A$ (either 1 or 5), and let $k = 1,2,3,4,5$

Using $ G_k(s) = E(s^{T_k}) $ as the probability generating function, we want to show that $$ G_3(s) = \frac{(p^2+q^2)s^2}{1-2pqs^2} $$

So, I'm kind of lost on how to start this. What is the first step to tackling a question like this?

StubbornAtom
  • 17,932
Kevin
  • 347
  • perhaps a pun was intended -- the answer is to apply "first step analysis". I.e. calculate $\mathbb E(s^{T_k}) = \mathbb E\big[ \mathbb E(s^{T_k}\vert X_1)\big]$. that puts you in state 2 or 4. And apply first step analysis there to be absorbed with probabilities p,q respectively or recover the original PGF (i.e. value at state 3). Conceptually you are looking for a generating function for the hitting time of absorbption, which was this thread https://math.stackexchange.com/questions/3521200/generating-function-for-the-hitting-time-of-a-markov-chain/3521464 – user8675309 Feb 05 '20 at 00:20
  • Thanks for the comment, i'm not too sure I understand you though. How does conditioning the expected value bring us to state 2 or 4, – Kevin Feb 05 '20 at 00:32
  • what if I asked about a simpler question of $\mathbb E(T_k) = \mathbb E\big[ \mathbb E(T_k\vert X_1)\big]$ -- could you relate $\mathbb E(T_k\vert X_1)$ to the expected times until absorbtion from states 2 and 4? – user8675309 Feb 05 '20 at 00:37
  • If our absorbing states are 2 or 4, wouldnt the expected time just be qs + ps? I dont know if I answered your question or not. – Kevin Feb 05 '20 at 00:40

0 Answers0