A matrix associated to a transition of a Markov chain. The entries of this matrix represents a probability with the sum of a whole column being $1$.
Questions tagged [transition-matrix]
292 questions
8
votes
2 answers
Distribution of $ad-bc$
I'm interested in the stochastic process
$$f_t=(ad-bc)_t$$
where $(a,b,c,d)_t$ is governed by the following transition rules:
$$\begin{align}(a,b,c,d) \rightarrow \begin{cases}
(a+1,b,c,d) \;\;\;\; \text{ with probability }p_x &\text{ if…
blue_egg
- 2,333
6
votes
3 answers
How can a markov transition matrix have eigenvalues other than 1?
A Markov transition matrix has all nonnegative entries and so by the Perron-Frobenius theorem has real, positive eigenvalues. In particular the largest eigenvalue is 1 by property 11 here. Furthermore in these notes (sec 10.3) it says that the…
blue_egg
- 2,333
5
votes
1 answer
How to interpret clusters on Markov chain time characteristics?
I have a complex network $G=(V,E)$ from multivariate financial time series in which a single vertex $v_i$ represents the types of states corresponding to the combination of the fluctuations of the prices on a given time frame, a single edge…
Nick
- 1,259
4
votes
1 answer
Find the return time of a state using absorption probabilities in a finite Markov chain
Suppose we have a Markov chain with transition matrix
$$\textbf{P}= \begin{pmatrix} \frac{1}{2} & \frac{1}{2} & 0&0&0&0 \\
\frac{1}{3} & \frac{2}{3} &0&0&0&0 \\\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&0 \\ \frac{1}{4} &…
user2545
- 311
4
votes
1 answer
Proof using strong Markov Property
Let $X = (X_n)_{n\in\mathbb{N}_0}$ be a homogenous Markov Chain with starting distribution $\mu$ and transition matrix $P$, where $P(x,x)<1$ for all $x\in S$ and
$\tau_0:=0$ and $\tau_{k+1}:=$ inf$\{n\geq \tau_k: X_n \neq X_{\tau k}\}…
Tino
- 137
4
votes
1 answer
Expected number of time steps to return to initial state
Suppose I take a deck of cards and remove all the cards with numbered ranks and only keep the jack, queen, king and ace cards (of the four ‘suits’ clubs, diamonds, hearts and spades). Starting with the four jacks, I play a random swapping game,…
TK99
- 427
4
votes
0 answers
Proof of the existence of a unique stationary distribution in a finite irreducible Markov chain.
I am currently trying to understand a proof for the above, which states that, in other words, there exists a unique $\overrightarrow{v}$, such that $\overrightarrow{v}P = \overrightarrow{v}$ for the transition matrix $P$.
He first shows that there…
zd_
- 742
4
votes
2 answers
Randomly swapping two balls in two urns with 3 balls each. In total $3$ are black and 3 are white. Is this process a Markov chain?
Three white and three black balls are distributed in two urns in such a way that
each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white balls. At each step, we draw one ball from each urn…
Win_odd Dhamnekar
- 1,075
- 2
- 11
- 25
3
votes
0 answers
Operator Norm Upper Bound via Policy Lipschitzness
Let $\mathcal{S}$ be the state space, and the transition kernel to be $K_i(s,ds') = \sum_{a} P^k(ds'|s,a)\pi_{{\theta_i}}(a|s)$.
How do I upper bound $\|K_1-K_2\|$. What I have tried is to follow the definition, that is
$$\|K_1-K_2\| =…
Leafstar
- 79
3
votes
0 answers
How can we construct the logical/arithmetic structure for this $\ell\times\ell$ transition matrix for DNA composite?
In a standard DNA system with $4$ bases ($\texttt{A}: 0, \texttt{T}: 1, \texttt{C}: 2, \texttt{G}: 3$), we enforce an $\ell$-RLL constraint (no more than $\ell$ consecutive identical bases, e.g., for $\ell= 2$, $\texttt{CCC}$ is forbidden) using a…
Dang Dang
- 320
3
votes
1 answer
Why does a $2\times2$ Markov matrix always have the eigenvector $[1, -1]$ associated with the eigenvalue that is not 1?
Consider a Markov transition matrix $ M $ for a population distribution between two cities. Let the initial state of the system be given by:
$$
u_0 = \begin{bmatrix} a - b \\ b \end{bmatrix}
$$
where $ a $ is the total population and $ b $ is the…
Patrickliu
- 33
3
votes
1 answer
Why is $\frac{\partial\boldsymbol{\Phi}\left(t, t_0\right)}{\partial t}=\mathbf{A}(t)\boldsymbol{\Phi}(t,t_0)$ where A(t) is the companion matrix?
I am stuck on the derivation of the identity in the title which is used in step three and four in the proof of
The Unique Solutions to Linear Nonhomogeneous Systems of First Order ODEs.
It is used in step three/four. Correct me if I am…
user1317687
3
votes
1 answer
Are finite state irreducible continuous Markov chains identifiable in general?
Let $S=\{1,...,h\}$ be a finite state space and $X(t)$ an irreducible Markov chain fully described by a generator matrix $Q$ with a transition probability matrix $P(t)=e^{Qt}$ on time horizon $[0,T]$. I have noticed in literature that…
user895064
3
votes
1 answer
Matrix representation of a linear map in other basis
Recently I was asked on my linear algebra course the following question:
Suppose the map $\varphi:L \rightarrow M$ has the matrix
$$A = \begin{bmatrix}1&-2&1\\1&-1&0\\2&-2&0\\-2&1&1\end{bmatrix}$$
in some pair of bases($e$, $f$). Can it have the…
Levon Minasian
- 1,182
3
votes
1 answer
Transition Matrix from B to C
If $B=\{ 2+x,1+2x\}$ and $C=\{ 1+x, 1-x\}$ are 2 basis for $P_1$, and $v=-3x+4$ find $[v]_B$, $_BP_C$ and $[v]_C$.
my attempt:
Since $B$ is a basis for $V$, then any $v\in B$ can be written "uniquely" as a linear combination of the vectors of…
Dima
- 2,501