2

Supposes I have a Discrete Time Markov Chain with Transition Matrix $P = \begin{pmatrix} p_{11} & p_{12} \\ p_{21} & p_{22} \end{pmatrix}$ and initial distribution $\begin{pmatrix} p^0_1 \\ p^0_2 \end{pmatrix}$.

In this case, we can define the Stationary Distribution $\begin{pmatrix} \pi_1 \\ \pi_2 \end{pmatrix}$ as:

$$\lim_{{n \to \infty}} \begin{pmatrix} \pi_1 \\ \pi_2 \end{pmatrix}^n = \begin{pmatrix} p^0_1 \\ p^0_2 \end{pmatrix} \begin{pmatrix} p_{11} & p_{12} \\ p_{21} & p_{22} \end{pmatrix}^n$$

My Question: Provided the Stationary Distribution does exist, is it possible to find out the number of iterations $n$ required for the Markov Chain to reach its Stationary Distribution within some distance $\epsilon$? Can someone please show me how to derive this formula for number of iterations?

Note: I recently learned about the Spectral Gap concept ... but I am not sure why it is relevant for this problem and where that formula comes from.

Thanks!

  • PS: I think if the Stationary Distribution = Limiting Distribution , then the number of iterations to reach either is the same. But in situations where the Limiting Distribution is not the same as the Stationary Distribution ... Is there a separate formula for finding out the number of iterations required to reach the Limiting Distribution?
stats_noob
  • 4,107
  • 7
    What makes you think that the stationary distribution can be reached in a finite number of steps? – jd27 Dec 15 '23 at 15:57
  • 1
    @ jd27; thank you so much for your reply! I think you are right ... by definition, it is impossible tor each the stationary distribution in a finite number of steps. – stats_noob Dec 15 '23 at 16:02
  • 1
    I wonder if it is possible to know: How many iterations (i.e. steps) are required for the stationary distribution to become "close enough" to the actual stationary distribution? i.e. how much will it deviate (epsilon/delta)? – stats_noob Dec 15 '23 at 16:05
  • 1
    yes it is possible to find convergence bounds based on the eigenvalues of the transition matrix. For a simple case this is done in the book "Probability Theory" by Achim Klenke in section 18.4. You can probably find information on this on the internet as well, one keyword would be "spectral gap". This is related as well. – jd27 Dec 15 '23 at 16:33
  • 1
    In some cases you do reach the stable distribution in a finite number of steps, like if all the entries are $\frac12$, but in the general case you don't. – Zoe Allen Jul 26 '24 at 12:57
  • I think before you can expect an answer you will need to define what precisely "close enough" means – jameselmore Jul 26 '24 at 18:03
  • Hi everyone thank you for your replies! – stats_noob Jul 27 '24 at 04:06
  • @jameselmore: [;ease see the update (I defined some tolerance level epsilon) – stats_noob Jul 27 '24 at 04:06
  • 1
    $\def\lpc{\lim_{n\rightarrow\infty}\pmatrix{\pi_1\\pi_2}}\def\lpr{\lim_{n\rightarrow\infty}\pmatrix{\pi_1&\pi_2}n}\def\tm{ \pmatrix{p{11} & p_{12} \ p_{21} & p_{22}}}\def\ivc{\pmatrix{p^0_1\ p^0_2}}\def\ivr{\pmatrix{p^0_1&p^0_2}}$ The notation in your equation $$\lpc^n=\ivc\tm^n$$ is incorrect. It needs to be either $$\lpr=\ivr\tm^n$$ (with $\tm$ being row-stochastic), or $$\lpc_n=\tm^n\ivc$$ (with $\tm$ being column-stochastic). I believe the first of these is now the more usual convention, although I think the latter is still used, and is quite common in older literature. – lonza leggiera Jul 27 '24 at 10:27
  • I don’t think that you’re interested in spectral gap but rather mixing time. Maybe you could look into these concepts and narrow down your question – LPZ Jul 27 '24 at 15:22
  • @stats_noob - there are many choices for distance measures, the boundary of epsilon is not as important as clarifying what measure you are using – jameselmore Jul 27 '24 at 16:24
  • @LPZ But isn't the spectral gap the primary factor affecting mixing time? – Zack Fisher Jul 27 '24 at 18:24
  • @ZackFisher yes the two are related but you can have cutoff phenomena etc. Plus, mixing time looked more like the approach taken by the OP with the alluded distance being the total variation – LPZ Jul 27 '24 at 19:39
  • thanks everyone for your kind replies ...do you think one of you can please post an answer if youhave time? – stats_noob Jul 30 '24 at 13:36

0 Answers0