0

Given a finite-state discrete time homogeneous Markov chain with N states and transition matrix $P$, a test for irreducibility is to determine whether this sum $$I + P + P^{2} + ... + P^{N-1} > 0$$ for all $(i,j)$ elements in that sum.

Does anyone have a reference for this test?

snoopy
  • 103
  • Welcome to MSE! Are you saying that this must be true for some $N$ or for all $N$? For the latter, notice that all the entries of $P$ are positive, what can you say about the entries of the sum $I+P+\dots+P^N$? – Kolakoski54 Nov 20 '24 at 20:06
  • Thank you for the welcome; for all N – snoopy Nov 20 '24 at 20:17
  • Note it goes only up to N-1 – snoopy Nov 20 '24 at 20:18
  • If all the (i,j) entries in the matrix that is the above sum are all positive then the Markov transition matrix P is irreducible. – snoopy Nov 20 '24 at 20:26
  • For $N=1$, it would mean that all the entries of the identity matrix are non zero, which is clearly false... we can prove that the Markov chain is irreducible if and only if it is true for some $N$. Note that it doesn't matter to go up to $N$ or to $N-1$. – Kolakoski54 Nov 20 '24 at 23:01
  • In an N=2-state Markov chain, the above sum is I + P. The sum is from 0 to N-1. Assume the minimum is always a 2-state MC. So, I'm still looking for a reference for this in the literature. – snoopy Nov 21 '24 at 01:39
  • There's a theorem of Wielandt, see https://math.stackexchange.com/questions/450090/if-p-is-a-regular-transition-probability-matrix-then-pn2-has-no-zero-ele – Gerry Myerson Nov 21 '24 at 03:04
  • 1
    Sorry for the confusion. I didn't find any reference for this result, but the proof seems accessible: all the entries of $P$ being $>0$, $(I+\dots+P^{N-1}){i,j}>0$ for all $(i,j)$ is equivalent to "for all $(i,j)$ there exists $k<N$ such that $P{i,j}^k>0$". So you would need to show that if a (nonzero probability) path exists between $i$ and $j$, then there exists one with length $<N$. Suppose for some $(i,j)$, the smallest's length was greater than $N$, such a path would visit some state twice and we would be able to find an even smaller one, and this would raise a contradiction. – Kolakoski54 Nov 21 '24 at 07:55
  • yes, indeed ... very nice Kolakoski54 – snoopy Nov 21 '24 at 10:27

0 Answers0