I am learning about Birth-Death Stochastic Processes (https://en.wikipedia.org/wiki/Birth%E2%80%93death_process):
$$P_{i,i+1}(\triangle t) = \lambda_i \triangle t + o(\triangle t), \quad i \geq 0$$ $$P_{i,i-1}(\triangle t) = \mu_i \triangle t + o(\triangle t), \quad i \geq 1$$
$$P_{i,i}(\triangle t) = 1 - (\lambda_i + \mu_i) \triangle t + o(\triangle t), \quad i \geq 1$$
$$ \begin{align*} \frac{dp_0(t)}{dt} &= -\lambda_0 p_0(t) + \mu_1 p_1(t), \\ \frac{dp_n(t)}{dt} &= \lambda_{n-1} p_{n-1}(t) - (\lambda_n + \mu_n) p_n(t) + \mu_{n+1} p_{n+1}(t) \quad \text{for } n \geq 1. \end{align*} $$
Here:
- $\lambda_n$ for $n \geq 0$ and death rates $\mu_n$ for $n \geq 1$,
- $\lambda_n$ is the rate of "births" or transitions from state $n$ to state $n+1$,
- $\mu_n$ is the rate of "deaths" or transitions from state $n$ to state $n-1$,
- $p_n(t)$ is the probability of being in state $n$ at time $t$.
Since a Birth-Death process is a continuous time stochastic process, I think we should be able to write it as an infinite state generator matrix.
The Birth Death process $\{X(t), t \geq 0\}$ is with state space $S = \{0, 1, 2, ...\}$ :
$$ Q = \begin{bmatrix} -\lambda_0 & \lambda_0 & 0 & 0 & \cdots \\ \mu_1 & -(\lambda_1 + \mu_1) & \lambda_1 & 0 & \cdots \\ 0 & \mu_2 & -(\lambda_2 + \mu_2) & \lambda_2 & \cdots \\ 0 & 0 & \mu_3 & -(\lambda_3 + \mu_3) & \cdots \\ \vdots & \vdots & \vdots & \vdots & \ddots \end{bmatrix} $$
And from here, write the Transition Matrix:
$$ P(t) = e^{Qt} $$
$$ \frac{d}{dt}P(t) = QP(t) $$
I have the following question: Suppose I know $\mu$ and $\lambda$ and the current state of this Birth-Death process. How can I find out the expected time when the process will first reach some state $k$?
Normally, in Markov Chains, I would use one of these two approaches (https://en.wikipedia.org/wiki/Absorbing_Markov_chain):
- First Step Analysis: The expected time until absorption $T_i$ the expected time until absorption starting from state $i$, and let $P_{ij}$ be the transition probability from state $i$ to state $j$. Then, the expected time until absorption satisfies the following system of equations:
$$T_i = 1 + \sum_{j \neq i} P_{ij} T_j$$
- Fundamental Matrix Approach: The transition matrix of a Markov Chain can be partitioned in the following way:
$$ P = \begin{bmatrix} Q & R \\ 0 & I \end{bmatrix} $$
Here, $Q$ is a sub-matrix that represents the transition probabilities between the transient states, $R$ is a sub-matrix that represents the transition probabilities from transient states to absorbing states, $I$ is a sub-matrix that corresponds to transitions between absorbing states and $0$ is a sub=matrix matrix of zeros that corresponds to transitions from absorbing states to transient states. The expected time to absorption is given by:
$$N = (I - Q)^{-1}$$
Thus, given some initial conditions, can any of these two approaches (i.e. first step analysis, fundamental matrix approach) be used to determine when the expected time required for a Birth-Death process to first reach a certain state?
Since the corresponding matrix of a Birth-Death process has infinite states, I am not sure if it can be partitioned as such. Since the corresponding matrix of a Birth-Death process has infinite states, the first step analysis approach might have infinite equations and I am not sure if a system of linear equations with infinite equations can be solved.
