3

Let ${X_n : n = 0, 1, 2, . . .}$ denote a Markov chain with the states $S = {1, 2, 3}$ and transition matrix P given by $$ \begin{bmatrix} 0 & 0.5 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.8 & 0.2 & 0 \end{bmatrix} $$

Determine whether the Markov chain has a marginal distribution. Determine the it and explain how you found it. Also determine whether the boundary distribution is unique.

So i know that an irreducible and aperiodic markov chain has a unique solution, which also is the limiting distribution. In this case, the stationary distribution is a limiting distribution. To find the stationary distribution i need to solve the following: \begin{align*} \pi_1 &= \frac{1}{2}\pi_2 + \frac{1}{2}\pi_3 \\ \pi_2 &= \frac{1}{10}\pi_1 + \frac{9}{10}\pi_3 \\ \pi_3 &= \frac{8}{10}\pi_1 + \frac{2}{10}\pi_2 \\ 1 &= \pi_1 + \pi_2 + \pi_3 \end{align*} I have solved it and got that $\pi_1 = \frac{1}{3}$, $\pi_2 = \frac{1}{3}$ and $\pi_3 = \frac{1}{3}$

Now that i have found my stationary distribution, which is also the limiting distribution what do i do with it? Is my limiting distribution the marginal distribution?

I know that the desired marginal distribution that i need to find is $(0.35, 0.25, 0.4)$ but i can't manage to get this.

Mohammed
  • 109
  • 1
    You need to solve $\pi = \pi P$ but you are solving $\pi = P\pi$. I believe "marginal distribution" is intended to mean the same thing as "stationary distribution" and indeed the solution to $\pi = \pi P$ is $\pi \approx (0.35, 0.25, 0.4)$. The first equation should be $$ \pi_1 = (0.1)\pi_2 + (0.8)\pi_3$$ – Michael May 23 '23 at 20:38
  • Mind me asking why the first equation should be that? I mean i thought that the first equation would be what i had because from state 1 we can go to both states 2 and 3 with a probability of 0.5 – Mohammed May 23 '23 at 20:42
  • 1
    If $X(t)$ is the DTMC we get $P[X(t+1)=j]=\sum_{i=1}^3 P[X(t)=i]P_{ij}$. In steady state it is $\pi_j = \sum_{i=1}^3 \pi_i P_{ij}$ and in matrix form $\pi = \pi P$ (where $\pi$ is a row vector). – Michael May 23 '23 at 20:44
  • Okay but the system of equations no longer have a solution this way? – Mohammed May 23 '23 at 20:45
  • 1
    I do not know what you mean. I repeat that you need to solve $\pi = \pi P$ (where $\pi$ is an unknown row vector) but you are solving $\pi = P\pi$ (where $\pi$ is an unknown column vector). A solution to more precision is $$\pi = (0.345991561181435 \quad 0.253164556962025 \quad 0.400843881856540)$$ – Michael May 23 '23 at 20:46
  • I'm thinking that i need to have one equation instead of four then that i need to solve or am i incorrect about that? – Mohammed May 23 '23 at 20:49
  • 1
    The matrix equation $\pi = \pi P$ involves three linear equations. I wrote the first of those three linear equations ($\pi_1=(0.1)\pi_2 + (0.8)\pi_3$). I did not write the other two. – Michael May 23 '23 at 20:50
  • I think i understand know. I have these equations instead of those i had before: $$ \begin{align} \pi_1 &= \frac{1}{10}\pi_2 + \frac{8}{10}\pi_3 \ \pi_2 &= \frac{1}{2}\pi_1 + \frac{2}{10}\pi_3 \ \pi_3 &= \frac{1}{2}\pi_1 + \frac{9}{10}\pi_2 \ 1 &= \pi_1 + \pi_2 + \pi_3 \end{align}$$ Is that correct – Mohammed May 23 '23 at 20:53
  • 1
    Yes, you can solve those and verify the solution. – Michael May 23 '23 at 20:55
  • 1
    Yeah it makes a lot more sense now and i can see it gives me the desired result so thanks a lot for the help! – Mohammed May 23 '23 at 20:57
  • 1
    Great. If you want you can type in your answer to your own question now, which is standard practice on this website when you can get an answer based on hints. – Michael May 23 '23 at 20:58
  • Yeah i will do so in a bit once i have managed to write my answer into latex. – Mohammed May 23 '23 at 21:04

1 Answers1

1

I have most of my information from the book Introduction to Probability, Statistics, and Random Processes by Hossein Pishro-Nik and will reference the pages i have looked at for the PDF version here and there!

We have that an irreducible and aperiodic Markov chain has a unique solution, which is also a limiting distribution of the Markov chain. In this case, if the chain is also aperiodic, we conclude that the stationary distribution is a limiting distribution (page 871).

By using RStudio you get the Markov chain is both irreducible and aperiodic and therefore has a unique solution, which is the stationary distribution, which is then a limiting distribution. The stationary distribution can be found by solving $\pi = \pi P$ which is just the following equations (Example 11.15 on page 872): \begin{align*} \pi_1 &= \frac{1}{10}\pi_2 + \frac{8}{10}\pi_3 \\ \pi_2 &= \frac{1}{2}\pi_1 + \frac{2}{10}\pi_3 \\ \pi_3 &= \frac{1}{2}\pi_1 + \frac{9}{10}\pi_2 \\ 1 &= \pi_1 + \pi_2 + \pi_3 \end{align*} After which you get the stationary distribution, which is also the limit distribution: $\pi_1 = 0.35$, $\pi_2 = 0.25$ og $\pi_1 = 0.40$.

Mohammed
  • 109