Birth-death process: What is the distribution of reached states before reaching an absorbing state?

Question

Intro

I am working on a birth-death process. For a given choice of parameter ($n=6$, $Wa=1$, $Wb=0.95$, see below), the transition matrix is

$$\left( \begin{array}{ccccccc} 1. & 0.144928 & 0 & 0 & 0 & 0 & 0 \\ 0. & 0.717391 & 0.229885 & 0 & 0 & 0 & 0 \\ 0 & 0.137681 & 0.551724 & 0.25641 & 0 & 0 & 0 \\ 0 & 0 & 0.218391 & 0.5 & 0.225989 & 0 & 0 \\ 0 & 0 & 0 & 0.24359 & 0.559322 & 0.140056 & 0 \\ 0 & 0 & 0 & 0 & 0.214689 & 0.726891 & 0. \\ 0 & 0 & 0 & 0 & 0 & 0.133053 & 1. \\ \end{array} \right)$$

, where a value at row $i$ and column $j$ is the probability to go from state $j$ to state $i$. In the markov process, there are $n+1=7$ possible states (denoted [0,1,2,...,11, 12]). The stationary distribution (given by the leading eigenvector)

This matrix can be obtained in Mathematica with

n = 6;
Wa = 1;
Wb = 0.95;
mat = Table[
   p = j/n;
   Wbar = Wa p + Wb (1 - p);
   pp = (Wa p)/Wbar;
   If[j == i, pp p + (1 - pp) (1 - p),
    If[j == i + 1, pp (1 - p),
     If[j == i - 1, (1 - pp) p, 0]]],
   {i, 0, n}, {j, 0, n}];

The stationary distribution of this birth-death process (given by the leading eigenvectors of mat) shows that there are two absorbing states, $i=0$ and $i=n+1=7$.

Question

Imagine you were to simulate this process a great number of times always starting from the state $i=1$ and stopping the simulation as soon as an absorbing state is reached. You would have spent a given amount of time on state $i$ and a given amount of time on any state. The time spent on any state is the expected time before reaching an absorbing state multiplied by the number of simulations. What is the relative frequency (relative to the total amount of time spent on any state) of times spent on any given state $i$?

Example

For example, imagine, we were to look for this answer numerically (while I am looking for an analytical solution) and we make (only) three runs.

• Run $1$ visits $15$ states (not counting the absorbing states), and visits state $2$ three times (so the frequency is 3/15 = 1/5).
• Run $2$ visits $5$ states, and visits state $2$ twice (so 2/5).
• Run $3$ visits $25$ states, and visits state $2$ just once (so 1/25)

The frequency spent on state $i=2$ is $\frac{3+2+1}{15+5+25}=\frac{2}{15}$

Answer 1

I believe this approach will work for your problem:

Modify the state transition matrix as follows. Since both states $0$ and $6$ are absorbing states, and you start every new run at state $1$, identify states $0$ and $6$ as state $0$, and have state $0$ go to state $1$ with probability $1$. That is, the new matrix will look like this:

$$ A = \left[ \begin{array}{cccccc} 0 & 0.144928 & 0 & 0 & 0 & 0.133053 \\ 1 & 0.717391 & 0.229885 & 0 & 0 & 0 \\ 0 & 0.137681 & 0.551724 & 0.256410 & 0 & 0 \\ 0 & 0 & 0.218391 & 0.500000 & 0.225989 & 0 \\ 0 & 0 & 0 & 0.243590 & 0.559322 & 0.140056 \\ 0 & 0 & 0 & 0 & 0.214689 & 0.726891 \end{array} \right] $$

Use standard techniques to solve for the equilibrium probability distribution vector $\vec{p} \equiv [p_0 \; p_1 \; p_2 \; p_3 \; p_4 \; p_5]^T$ in $A\vec{p} = \vec{p}$. Then for any $i, 1 \leq i \leq 5$, $p_i/(1-p_0)$ gives the relative frequency of state $i$ over all runs.