2

The question relates to using method of generating function for finding $n$th power of a transition matrix $P$

In the Text Book the generating function has been defined as:

$P(s)= 1+sP+s^2P^2+s^3P^3+\cdots +s^n P^n$ where $|s|<1$

($s$ is a variable of function $P(s)$)

As $n\to\infty$, $s^nP^n\to0$

I have not understood how from this step we conclude that $P(s)$ is the inverse of matrix $I-sP$.

SAK
  • 539
  • 1
  • 5
  • 21

1 Answers1

2

If $P\in\mathbb R^{N\times N}$ is a stochastic matrix, then for any $x\in\mathbb R^N$ we have \begin{align} \|Px\|_1 &=\sum_{i=1}^N\sum_{j=1}^N P_{ij}x_j\\ &=\sum_{j=1}^N\sum_{i=1}^N P_{ij}x_j\\ &=\sum_{j=1}^Nx_j\sum_{i=1}^N P_{ij}\\ &=\sum_{j=1}^N x_j\\ &= \|x\|_1. \end{align} It follows that $$\|P\|_1 = \sup_{\|x\|_1=1}\|Px\|=1. $$ From submultiplicativity we have $$\|P^n\|_1\leqslant\|P\|_1^n=1 $$ for all $n$, so if $|s|<1$, then $$\lim_{n\to\infty}\|sP\|^n_1 \leqslant \lim_{n\to\infty}|s|^n\|P\|_1^n = \lim_{n\to\infty}|s|^n=0. $$ This implies that the generating function $$P(s) = \sum_{n=0}^\infty (sP)^n $$ converges. Letting $P_n := \sum_{k=0}^n (sP)^k$, we see that $$\lim_{n\to\infty} (I-sP)P_n = \lim_{n\to\infty} \left(\sum_{k=0}^n (sP)^k - \sum_{k=0}^{n+1} (sP)^k\right) = \lim_{n\to\infty} \left(I-(sP)^{n+1}\right) = I $$ (similarly, $\lim_{n\to\infty} P_n(I-sP)=I$), and hence $P(s) = (I-sP)^{-1}$.

Math1000
  • 38,041
  • 1
    That was very good. Text book had not given these steps. I have understood now. Thank you very much – SAK Sep 17 '16 at 19:41