17

I have two books on stochastic processes.

In one book, it says that the limiting matrix is possible to find if the matrix is regular, that is, if for some $n$ $P^n$ has only positive values. The other book says that the limiting values are possible to find if the Markov chain is recurrent, irreducible and aperiodic; it is then called ergodic.

Does this then hold:

aperiodic + irreducible $\Leftrightarrow$ ergodic $\Leftrightarrow$ regular?

And is there any difference whether it is a finite-state chain or not?

user119615
  • 10,622
  • 6
  • 49
  • 122

1 Answers1

19

For a finite MC it holds that

aperiodic + irreducible $\Leftrightarrow$ ergodic $\Leftrightarrow$ regular

as you expected. For an infinite MC it holds that

aperiodic + irreducible + positive recurrent $\Leftrightarrow$ ergodic,

and being "regular" in the infinite setting would require a more precise definition.

................................ explanations following ................................

For every finite or inifinite Markov chain (MC) it holds that

$aperiodic + irreducible + positive~recurrent \Leftrightarrow ergodic$.

See for example here for a proof. For every finite MC, irreducibility already implies positive recurrence, see here for a proof.

Further, for every finite MC we have that

$aperiodic + irreducible \Leftrightarrow regular$.

Proof sketch: the definition of a finite irreducible MC gives that $\forall i, j \in \Omega : \exists k > 0 : P^k[i,j] > 0$. However, there might be no $k$ such that all entries are simultaneously positive - due to periodicities. But if the chain is additionally aperiodic, it follows that $\exists k > 0 : \forall i, j \in \Omega : P^k[i,j] > 0$, which matches your definition of being regular.

Finally, I don't see a canonical way how you would generalize the property "regular" to infinite Markov chains. So, I just ignore the term "regular" for infinite chains here.

Community
  • 1
cubic lettuce
  • 1,080
  • 9
  • 17
  • Thank you very much!, it cleared the situation up for me! – user119615 Jan 21 '14 at 12:01
  • Are you sure that the "+⇔" part is correct? What raises my concerns, is the following sentence on p.598 in Murphy, Machine Learning: A Probabilistic Perspective, 2012: "One can show that sufficient conditions* to ensure regularity are that the chain be irreducible (singly connected) and that every state have a self-transition."* The self-transition implies aperiodicity, but the condition is entitled as only sufficient, but not necessary. – kostrykin Jul 22 '22 at 23:21
  • It is not necessary to have self-transitions, that's probably what the book means. – cubic lettuce Jul 25 '22 at 06:59
  • Regarding the last point, my first guess would be to explore countably infinite Markov processes as a limit of finite Markov processes, each of which is regular? – MRule Dec 18 '23 at 11:05