Let $X = (X_n)_{n\in\mathbb{N}_0}$ be a homogenous Markov Chain with starting distribution $\mu$ and transition matrix $P$, where $P(x,x)<1$ for all $x\in S$ and
$\tau_0:=0$ and $\tau_{k+1}:=$ inf$\{n\geq \tau_k: X_n \neq X_{\tau k}\} (k\in\mathbb{N}_0)$.
How can I show with the strong Markov Property that the sequence $Y=(Y_k)_{k\in\mathbb{N}_0}$ with $Y_k:=X_{\tau k} (k\in\mathbb{N}_0)$ is a homogenous Markov Chain? What are then the starting distribution and the transition matrix?
I first state the Strong Markov Property, which is a direct consequence of the Markov Property in discrete time (note : my version may differ from yours : if it is a stronger statement, then prove it from your version).
Let $X_i$ be a Markov chain on state space $S$ and $T$ be an almost surely finite stopping time adapted to $X$. Then, for each $x_0,...,x_T$ in the state space and $A$ subset of the state space so that the LHS is well defined, $$P(X_{T+1} \in A |X_0 = x_0,...,X_T = x_T) = P(X_{T+1} \in A | X_T = x_T) = P(X_1 \in A | X_0 = x_T)$$
Basically, this means that any choice of stopping time, the future following the stopping time depends only on the value at the stopping time and not on the values before this stopping time. Furthermore, by homogeniety, the dependence(i.e. the conditional probability) is exactly the same as for the Markov process itself.
Even shorter : A homogeneous discrete time Markov process renews itself at every stopping time.
Note that similar to a usual Markov chain, we have the extension : for a function $g$ of $N$ variables on the state space, $$E(g(X_{T+1} ,X_{T+2},X_{T+N}) |X_0 = x_0,...,X_T = x_T) = E(g(X_{T+1},x_{T+2},...,x_{T+N}) | X_T = x_T) = E(g(X_1,X_2,...,X_N) | X_0 = x_T)$$
By using $g$ as indicator functions we recover the usual definition.
To now prove the statement , we first note that $\tau_0$ is a stopping time, for obvious reasons. Each $\tau_k$ is almost surely finite as a random time: remember that because $P(x,x) < 1$, you can use Borel-Cantelli to show that almost surely, if the MC starts at $x$ it will exit $x$ in finite time. It is a stopping time : that can be shown inductively by seeing that for any $N$, we have : $$\{X_{\tau_{k+1}} \leq N\} = \cup_{k=0}^N[\{X_\tau =k\} \cup_{i=k}^N \{X_i \neq X_{k}\}]$$
Therefore, if $\tau_k$ is a stopping time, so is $\tau_{k+1}$. We have by induction that each $\tau_k$ is a stopping time. Use the condition $P(x,x) < 1$ to see that $\tau_1 < \infty$ almost surely.(Hint : Consider the events $X_1 = x,X_2=X_1=x,X_3= X_2=X_1=x$ all conditioned on $X_0 = x$ for some $x$. The probabilities of these events can be written like a geometric series,which is summable because $P(x,x) < 1$. Hence by Borel Cantelli one of them almost surely won't happen, which is equivalent to $\tau_1$ being finite).
Use the SMP now : for example, $P(X_{\tau_k+1} = X_{\tau_k} | X_{\tau} =x) = P(X_1=X_0 | X_0 = x)$, now use Borel Cantelli on the series of events $X_{\tau_k+1} = X_{\tau_K},X_{\tau_k + 2} = X_{\tau_k+1} = X_{\tau_k}$ conditioned on $X_{\tau_k} = x$ and so on, to see that if $\tau_k$ is a.s. finite so is $\tau_{k+1}$. It follows that each $\tau_k$ is a.s. finite. This is required to define $Y_k$. (What is $Y_k$ if $\tau_k = \infty$? Undefinable, that's what).
We must now show that $Y_k$ is a Markov process. To see this, note that $Y_k = X_{\tau_k}$ is a well defined random variable almost surely, so we can speak of conditioning it and so on. Start with the definition, with the LHS non-zero. $$ P(Y_{k+1} = y | Y_{0} = y_0,...,Y_k = y_k) \\= P(X_{\tau_{k+1}} = y | X_0 = y_0,...,X_{\tau_k} = y_k) $$
By definition, $\tau_k$ is the smallest index after $\tau_{k+1}$ such that the value of $X_{\tau_k}$ is different from the value at $X_{\tau_{k-1}}$. It follows that for every index $M$ between $\tau_{k-1}$ and $\tau_k$, the value $X_M$ equals the value $X_{\tau_{k-1}}$.
That is, we have: $$ P(X_{\tau_{k+1}} = y | X_0 = y_0,...,X_{\tau_k} = y_k) \\ = P(X_{\tau_{k+1}} = y | X_0 = y_0,X_1 = y_0,...,X_{\tau_1 - 1} = y_0,X_{\tau_1} = y_1,X_{\tau_1+1}= y_1,...,X_{\tau_2} = y_2,...,X_{\tau_k - 1} = y_{k-1},X_{\tau_k} = y_k) $$
Now we can use the definition (note : the sum over the state space may be an integral, but nothing changes in the computation) $$ P(X_{\tau_{k+1}} = y | X_0 = y_0,X_1 = y_0,...,X_{\tau_1 - 1} = y_0,X_{\tau_1} = y_1,X_{\tau_1+1}= y_1,...,X_{\tau_2} = y_2,...,X_{\tau_k - 1} = y_{k-1},X_{\tau_k} = y_k) \\ = \sum_{N = 1}^\infty \sum_{x \in S}^{x \neq y_k} P(X_{\tau_k + 1} = y_k,X_{\tau_k + 2} = y_k,...,X_{\tau_K + N-1} =y_k, X_{\tau_k + N} = x |X_0 = y_0,X_1 = y_0,...,X_{\tau_1 - 1} = y_0,X_{\tau_1} = y_1,X_{\tau_1+1}= y_1,...,X_{\tau_2} = y_2,...,X_{\tau_k - 1} = y_{k-1},X_{\tau_k} = y_k ) $$
Each term inside the summation simplifies, thanks to the SMP, to $P(X_{\tau_k + 1} = y_k, X_{\tau_k + 2} = y_k,...,X_{\tau_k+N-1} = y_k,X_{\tau_k+N} = x | X_{\tau_k} = y_k) = P(X_1 = ... = X_{N-1} =y_k,X_N = x | X_0 = y_k) = P(y_k,y_k)^{N-1} P(y_k,x)$.
Sum this from $N= 1$ to $\infty$ (you can switch the summations because the summands are non-negative) to get $\frac{P(y_k , x)}{1-P(y_k,y_k)}$.
Thus, we get : the probability that $P(Y_{\tau_k + 1} = x | Y_{\tau_k} = y) = \frac{P(y,x)}{1-P(y,y)}$ if $x \neq y$, and the probability is $0$ if $x = y$.
Use the calculations I made to show that $Y_{k}$ is a Markov process. Get the starting distribution from $Y_0 = X_0$ and the transition probabilities as mentioned.
