2

Suppose we flip a fair coin repeatedly until we have flipped four consecutive heads. What is the expected number of flips that are needed? The hint is given is as follows: Consider a Markov chain with state space {0,1,...4}.

Can anybody give an idea of where to start? Thanks! As I make progress I will add what my solution becomes. Otherwise any help would be appreciated.

EhBabay
  • 740
  • 1
    I have given a solution here of the same problem with $5$ consecutive. In order to make the argument accessible to people not familiar with Markov chains, the solution did not mention them explicitly, though you will recognize transition probabilities. – André Nicolas Oct 23 '14 at 15:19
  • I see! Thank you. I will still post my solution. – EhBabay Oct 23 '14 at 15:20
  • You are welcome. Other solutions there use other techniques. There is a Markov chain solution somewhere on MSE, but my search skills are limited. – André Nicolas Oct 23 '14 at 15:22

2 Answers2

2

Indeed let $X_n$ denote the number of consequtive heads that have flipped at time $n$. The possible states of $X_n$ are $\{0,1,2,3,4\}$ since if $4$ consecutive heads the process is considered to end. Given $X_n=j$ the next state can be either $0$ if we flip a tail or $j+1$ if we flip another consecutive head, thus $$X_{n+1}|X_n=j=\begin{cases} 0, &\text{ with probability } \frac{1}{2} \\ j+1, &\text{ with probability } \frac{1}{2} \end{cases}$$ since each event (head or tails) occurs with the same probability. Hence, the transition matrix of the markov chain is given below $$\begin{pmatrix}\frac{1}{2}&\frac{1}{2}&0&0&0\\\frac{1}{2}&0&\frac{1}{2}&0&0\\\frac{1}{2}&0&0&\frac{1}{2}&0\\\frac{1}{2}&0&0&0&\frac{1}{2}\\0&0&0&0&1\end{pmatrix}$$ where we have assumed that as long the game reaches state $4$ i.e. 4 consecutive heads the process seizes. The initial state is $X_0=0$. Let $h(j)$ denote the expected number of flips until you reach state $4$. Then you have the following system of equations $$\begin{align*}h(0)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(1)\\h(1)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(2)\\h(2)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(3)\\h(3)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(4)\\h(4)&=0\end{align*}$$ Now solve the above system recursively to obtain $h(0)$ which is the expected number of flips needed.

The solution is $h(0)=30$.

Jimmy R.
  • 36,148
1

Here is my solution!

$P=\begin{bmatrix} 1/2 &1/2 &0 &0 &0 \\ 1/2&0 &1/2 &0 &0 \\ 1/2&0 &0 &1/2 &0 \\ 1/2&0 &0 &0 &1/2 \\ 0&0 &0 &0 &1 \end{bmatrix}$

We will then treat state 4 like a recurrent state and solve for the matrix of the form

$\begin{bmatrix} I & & &0 \\ & & & \\ & & & \\ S& & & Q \end{bmatrix}$ Note: Sorry I don't know how to partition the matrix into four equal parts.

We then solve for the matrix $M=(I-Q)^{-1}$. So we find that $M=\begin{bmatrix} 16 &8 &4 &2 \\ 14&8 &4 &2 \\ 12& 6 & 4 &2 \\ 8&4 &2 &2 \end{bmatrix}$

We then take the matrix M and add up the first row 16+8+4+2=30.

EhBabay
  • 740