4

Suppose I have a circular markov chain. At each state, you are equiprobable to transition to the state immediately to your left or right.

I'm interested in the probability of visiting the jth state last? That is to say, what is the probability of reaching all the other states at least once before the jth state?

Demetri Pananos
  • 3,294

1 Answers1

2

If the number of states is $N\ge 3$, then the answer is $1/(N-1)$, independent of the starting point and of $j$.

Setup: Let the states be $0,\dots, N-1$ and WLOG assume we start from $0\ne j$. Let $I_j$ be the ``interval" consisting all elements in the cycle, except for $j$ and its two immediate neighbors, $j\pm 1 \mod N$.

We get to $j$ last if and only if we

  1. exit $I_j$,

and then

  1. If exiting $I_j$ from $j\pm 1 \mod N$, then we need to exit the interval consisting of all elements but $j$ from $j\mp 1 \mod N$.

The first event occurs with probability $1$, and, due to symmetry, the probability of the event on part 2. is independent of the past, and is equal to the probability that symmetric random walk on $Z$ starting from $1$ hits $N-1$ before hitting $0$. This, by a simple calculation (the probability as a function of the starting point satisfies a linear recurrence with boundary values), is equal to $1/(N-1)$.

Fnacool
  • 9,581