Suppose we have a time-homogeneous chain with a state space with $d$ states, where $d\in\mathbb N^+$. The transition matrix $P$ is given as below, which is a $d\times d$ matrix.
$ P= \begin{bmatrix} \frac{3}{4} & \frac{1}{4} & 0 & 0 & 0 & ... & 0 & 0 & 0\\ \frac{1}{4d} & \frac{3}{4} & \frac{d-1}{4d} & 0 & 0 & ... & 0 & 0 & 0\\ 0 & \frac{2}{4d} & \frac{3}{4} & \frac{d-2}{4d} & 0 & ... & 0 & 0 & 0\\ ... & ... & ... & ... & ... & ... & ... & ... & ...\\ 0 & 0 & 0 & 0 & 0 & ... & \frac{d-1}{4d} & \frac{3}{4} & \frac{1}{4d}\\ 0 & 0 & 0 & 0 & 0 & ... & 0 & \frac{1}{4} & \frac{3}{4} \end{bmatrix}$
I want to show that all states are persistent.
So, I try to calculate $f_{ij}(n)$, the probability of returning to state for the first time, given we start in state in steps. But it turns out that for large $n$, $f_{ij}(n)$ becomes very complicated and contains many terms (due to the fact that we can stay at many different states during the process). I was somehow stuck here. Thanks for help on showing that all states are persistent by direct calculation.
I know a way to prove that all states are persistent, but it is not based on much calculation. Here is the reasoning: We know the chain is irreducible from $P$. It is finite-state. We are done. However, I want to prove the result by direct calculation/its definition. Thanks!
