If I flip a fair 6-sided die I win the amount of money equivalent to the number that shows up. I can play a maximum of three times but have to decide whether to stop or continue with each throw and would win that particular throws face value. How much should I be willing to pay for this game?
Thanks
Let $X_1, X_2, X_3$ be three iid random variables s.t. $\mathbb{P}\{X_i = k\} = 1/6, \ k \in \{1,2,3,4,5,6\}$.
$$\mathbb{E}X_i = 3.5$$
$$\mathbb{P}\{X_{i+1} \geq X_i|X_i = x\} = \sum_{k = x}^{6} \mathbb{P}\{X_{i+1} = k\} = (6-x+1)/6$$
$$\mathbb{P}\{\max(X_{i+1}, X_{i+2}) \geq X_i|X_i = x\} = \sum_{k = x}^{6} \mathbb{P}\{\max(X_{i+1}, X_{i+2}) = k\} = \sum_{k=x}^{6} \frac{2k-1}{36} = \frac{(6-x+1)(6+x-1)}{36}$$
With $p=0.5$, the optimal strategy would be,
Let,
$$p_1 = \mathbb{P}\left\{\frac{(6-X_1+1)(6+X_1-1)}{36} < p\right\} = \mathbb{P}\{X_1^2-2X_1-17 > 0\} = \mathbb{P}\{X_1 > 1+3\sqrt{2}\} = \frac{1}{6}$$
$$p_2 = \mathbb{P}\left\{\frac{6-X_2+1}{6} < p\right\} = \mathbb{P}\{X_2 > 7-6p\} = \mathbb{P}\{X_2 > 4\} = \frac{1}{3}$$
$$p_3 = \mathbb{P}\left\{\frac{(6-X_1+1)(6+X_1-1)}{36} \geq p\ \wedge \frac{6-X_2+1}{6} < p\right\} = (1-p_1)p_2$$
$$p_4 = \mathbb{P}\left\{\frac{(6-X_1+1)(6+X_1-1)}{36} \geq p\ \wedge \frac{6-X_2+1}{6} \geq p\right\} = (1-p_1)(1-p_2)$$
$U$ being the net utility. The expected utility is,
$$ \begin{align} \mathbb{E}U &= \mathbb{E}\left(X_1\bigg|\frac{(6-X_1+1)(6+X_1-1)}{36} < p\right)p_1 \\ &+ \mathbb{E}\left(X_2\bigg|\frac{(6-X_1+1)(6+X_1-1)}{36} \geq p\ \wedge \frac{6-X_2+1}{6} < p\right)p_3 \\ &+ \mathbb{E}\left(X_3\bigg|\frac{(6-X_1+1)(6+X_1-1)}{36} \geq p\ \wedge \frac{6-X_2+1}{6} \geq p\right)p_4 \\ &= 6 \cdot p_1 + \frac{11}{2} \cdot p_3 + \frac{7}{2} \cdot p_4 = 1 + \frac{55}{36} + \frac{70}{36} \approx 4.47 \end{align} $$
