This makes sense to me intuitively, since infinite return time if null recurrent (or transient i.e positive probability of no return) doesn't seem to be possible if there are finite states.
I found a convincing argument here, but I'm not able to prove this mathematically. Could someone please provide hints/outline for a proof?
Thanks in advance
for an algebraic proof, note that if your chain had no positive recurrent class then that means it decomposes purely into null recurrent and transient classes.
This implies $P^k\to \mathbf 0$
using the Frobenius norm for convenience, this is equivalent to
$\big\Vert P^k-\mathbf 0\big \Vert_F \lt \epsilon$
for any $\epsilon \gt 0$ for $k$ large enough.
but $P\mathbf 1 = \mathbf 1$ so $\lambda_1=1$ is an eigenvalue of $P$. Applying Schur Triangularization to $P$ we get
$1 =\lambda_1^k \leq \big\Vert P^k-\mathbf 0\big \Vert_F $
for all $k$ so $P^k\to \mathbf 0$ is impossible and $P$ must contain a positive recurrent class.
wordsmithing note to OP: it isn't actually true that all finite state markov chains 'are' positive recurrent, it is true that they must contain a positive recurrent class. (To falsify the former, e.g. construct a simple absorbing state chain where absorbtion happens WP1.) Alternatively you could say that all irreducible finite state markov chains are positive recurrent.
