1

Alice rolls a fair 6 − 6−sided die with the values 1 − 6 on the sides. She sees that value showing up and then is allowed to decide whether or not she wants to roll again. Each re-roll costs $1. Whenever she decides to stop, Alice receives a payout equal to the upface of the last die she rolled. Note that there is no limit on how many times Alice can re-roll. Assuming optimal play by Alice, what is her expected payout on this game?

I tried using a similar technique as the answer posted in this thread:The expected payoff of a dice game

My line of reasoning is as follows:

  1. Calculate the expected value for each round:
  • So round 1 is $\frac{1+2+3+4+5+6}{6} = 3.5$
  • Round 2: $\frac{0+1+2+3+4+5}{6} = 2.5$
  • Similarly I get round 3 = $1.5$, round 4 = $0.5$ and then any subsequent rounds have negative expected value so we don't consider them.

  1. Then like in the thread attached, I worked backwards from round 4 - if round 4's expected value is $0.5$, then in round 3, we need to roll a 3,4,5 or 6, otherwise we go to round 4 i.e. our expected value for round 3 and 4 is: $\frac{4}{6} *\frac{(1+2+3+4)}{4} + \frac{2}{6} *0.5 = \frac{11}{6}$. That means in round 2 we need to get higher than 11/6 i.e. we need a $3,4,5 or 6$ (as there is a $1 cost).

  2. Repeating this process, I got an answer of 53/18 for the expected value of rounds 2, 3 and 4.

  3. Finally after doing this another time, I got that the expected payout of this game is $\frac{215}{54} = 3.98148...$.

However the correct answer is exactly 4 so it seems I am slightly off somewhere.

Apex345
  • 363
  • 1
  • 9
  • 2
    Any sensible strategy is essentially going to be: take the money if you roll $k$ or more but pay the $$1$ and keep rolling if you roll less than $k$ and continue to do this until you get $k$ or more. So you need to test this for $k \in {1,2,3,4,5,6}$ and find which has the highest expected return. $4$ is indeed the maximum expected return if the initial roll is free, though there is more than one $k$ which gives this. – Henry Jul 29 '24 at 18:37

1 Answers1

1

A sensible strategy is going to one where you take the money if you roll $k$ or more but pay the $\$1$ and keep rolling if you roll less than $k$ and continue to do this until you get $k$ or more, because if it is worth paying to reroll with a score less than $k$ now, it will be worth doing so with a score less than $k$ in the future too while if it worth taking the money now with a score of $k$ or above then it will be worth taking the money in the future with a score of $k$ or above.

With such a strategy, if the expected overall gain is $E_k$, and since either you roll $k$ or more with probability $\frac{7-k}{6}$ and then take an expected gain of $\frac{k+6}{2}$ or you roll less than $k$ with probability $\frac{k-1}{6}$ and pay $1$ for a future expected gain of $E_k$, then you can say $$E_k = \left(\frac{7-k}{6}\right)\left(\frac{k+6}{2}\right)+\left(\frac{k-1}{6}\right)(-1+E_k)$$ which, avoiding infinite sums, simplifies to $$E_k=\frac{44-k-k^2}{14-2k}.$$

This means $E_1=3.5$, $E_2=3.8$, $E_3=4.0$, $E_4=4.0$, $E_5=3.5$, and $E_6=1.0$. Of these $E_3$ and $E_4$ have the highest values. So the strategy that maximises the expectation is:

  • If you roll any of $4,5,6$ then take the money
  • If you roll $1$ or $2$ then pay $\$1$ and start again
  • If you roll $3$ do whichever you prefer

and this gives an overall expectation of $\$4$.

Henry
  • 169,616
  • Thanks for your response. The only bit I don't get is what do you mean by: 'if it is worth paying to reroll with a score less than k now, it will be worth doing so with a score less than k in the future too'. How do you know this is true, because I thought to use this strategy for this question https://math.stackexchange.com/questions/179534/the-expected-payoff-of-a-dice-game but it turned out that you had to take the expected value of all future rounds into consideration to determine your cutoff for the current round. – Apex345 Jul 30 '24 at 08:06
  • @Apex345 At any point, the expected value if you decide to pay $1$ and reroll with this strategy is $-1 + E_k$, and this takes all potential future rounds into consideration. – Henry Jul 30 '24 at 08:56
  • Ahhh right. So in the linked question, the reason we had to explicitly calculate the expected value of all future rounds is because there were only a finite number of rounds? I think i get it now. Thank you so much! – Apex345 Jul 30 '24 at 09:19
  • also is $E_k$ also dependent on the round number because let's say we are at round 10, the expected value of the game for rounds 11 onwards is going to be less than if we started at round 1, but round 10 relative to the subsequent rounds will require the same cutoff as playing the game from round 1, right? How were you able to deduce this quickly - for me this required a lot of thought? – Apex345 Jul 30 '24 at 09:47
  • @Apex345 If you are at round ten and have already paid your $$1$ for a reroll, then looking forward (so ignoring the $$10$ you have paid so far for ten rerolls), you have exactly the same expectation looking forward as you did at the very start. This is quite a common approach for questions where you are allowed to start again. – Henry Jul 30 '24 at 11:27