1

Inspired by this question, I want to know if there is a version of the scenario that actually fits Newb's intuition about the problem.

Scenario template

You roll a 6-sided die and add up the cumulative sum of your rolls. The game ends under the following conditions, with the associated payouts:

  • You choose to stop rolling: you win, and your payout is your cumulative total so far.
  • You roll a losing number: you lose, and your payout is 0.

The set of losing numbers is the part of the scenario that can vary.

Question 1: Newb's intuition

In the original question, Newb wrote:

Is there some integer N after which the answer to this question is always to roll?

I think that there is such an integer....My reasoning is that the [losing] numbers become sufficiently sparse for the expected value to always be in increased by rolling the die again.

The current top-rated answer demonstrates that this intuition is incorrect; the losing numbers do not grow fast enough to lead to this paradox.

But just because the losing numbers are not scarce enough in this particular case does not mean that there is no scenario for which this intuition is correct. So:

What set of "losing" numbers causes the expected value for continuing to roll to always exceed the expected value for stopping?

Question 2: The strategy

In a parenthetical, Newb wrote:

[This is] counter-intuitive, since this profit-maximizing strategy invariably results in going home with nothing.

Obviously a strategy with an expected payout of 0 is not actually profit-maximizing. So in this scenario, What is the optimal strategy?

  • I just came across this for the first time, now. Very interesting! Thanks for the good question + answer. – Newb Apr 20 '15 at 07:05
  • @Newb Thanks! Unfortunately I never got around to really finishing the answer to my satisfaction, and the question never garnered much interest anyway. Oh well. – Kyle Strand Apr 20 '15 at 15:24

1 Answers1

2

I wrote this more or less specifically so I could talk about Schelling points/fences, but it took forever to write the first section, so I'll flesh out the second section later.

The scenario

EDIT: According to the comment I wrote soon after writing this (see below), there's an arithmetic error somewhere in this section. Unfortunately, I no longer remember what it is and can't be bothered to find it now.

We want to define a sequence of numbers such that the following is true:

Letting $c_t$ be the cumulative total after $t$ rolls,and $f(n)$ be the $n^{th}$ "losing" number, a choice occurs whenever

$$ d(t) := f(n) - c_t \leq 6 $$

Let $P_{Lxy}(x,y):=P(L|d(t)=x|roll(y))$ be the probability of losing after $y$ rolls starting with a situation where the cumulative total so far is $x$ less than the next losing number. Note that $P_{Lxy}(x,y)=P_{Lxy}(x,y+1)$ is always (trivially) true away from the threshold ($P_{Lxy} = 0$) and true for $y \geq x$ near the threshold. The only "interesting" values of $y$, therefore, are $1$ (for the probability of losing after a single roll) and $y$ (for the total probability of losing before crossing the threshold, assuming you do not decide to quit before then). Thus, let $P_L(x):=P_{Lxy}(x,x)$, and note that $P_{Lxy}(x,1)=P_{Lxy}(1,1)=P_L(1)$ for $x\leq 6$.

If you successfully pass threshold $f(n)$, you will keep rolling until you have reached at least $f(n+1)-6$, i.e., the next decision threshold. The worst case is that you reach precisely $c_t = f(n+1)-6$; thus $f(n+1)-6$ is your minimum expected payout for crossing the threshold. (You may reach up to a payout of $f(n+1)-1$ without actually encountering a decision point, but I will ignore this in my analysis.)

In order for the "optimal" choice to be "continue rolling," the probability of crossing the next threshold (i.e. getting $c_{t+x} > f(n)$ for some number of rolls $x$) must exceed $c_t$ for all $n$, $t$ such that the above formula holds. In other words, in order for the decision at every threshold $f(n)$ to have a greater expected payout for continuing than for ending the game, the following (strict) inequality must hold:

$$ P_{L}(x) \cdot (f(n+1)-6) > f(n)-x $$

The worst-case scenario for a given threshold is when $x = 6$, because it provides the highest number of chances to lose: If $x = 1$, a single non-losing roll gets you past the threshold, but when $x = 6$, you could roll $1$ five times before finally rolling something else to cross the threshold. Each roll is a separate decision point, and each roll has the same probability ($= \frac{1}{6}$) of causing an instant loss.

Let $P_R(x)=\frac{1}{6}$ be the probability of rolling $x$ on a single roll.

In every case near the threshold ($d(t) \leq 6$), $P_L(1)=P_R(x) = \frac{1}{6}$. Note that $P_L(0)$ represents the probability of losing when the cumulative total actually hits a losing number, i.e. $1$.

The total probability of losing before crossing the threshold when $d(t) = 6$ is $P_L(6)$.

Recursively, for $x \leq 6$,

$$ P_L(x) = P_R(1)P_L(x-1) + \cdots + P_R(x-1)P_L(1) + P_R(x)P_L(0) \\ = \frac{1}{6}\sum_{i=0}^{x-1}P_L(i) \\ = \frac{1}{6}\sum_{i=0}^{x-2}P_L(i) + \frac{P_L(x-1)}{6} \\ = P_L(x-1) + \frac{P_L(x-1)}{6} \\ = \frac{7}{6}P_L(x-1) $$

Note that this holds for $6 \geq x>1$; $P_L(1) = \frac{1}{6}$, so

$$ P_L(x) = \frac{1}{6}(\frac{7}{6})^{x-1} $$

for $0<x \leq 6$.

Thus, the maximum probability of losing at any single threshold is:

$$ P_L(6) = \frac{16807}{46656} \approx 36 \% $$

And therefore, the above inequality becomes, in the worst case of $x=6$,

$$ \frac{16807}{46656} \cdot (f(n+1)-6) > f(n)-6 \\ (f(n+1)-6) > \frac{46656}{16807} \cdot (f(n)-6) \\ f(n+1) > \frac{46656}{16807} \cdot f(n) - \frac{179094}{16807} $$

(The coefficient looks suspiciously like $e$, but I think that's a coincidence.)

Therefore, any sequence of "losing numbers" that fits this inequality (note that it implies exponential rather than geometric growth) will create the observed paradox. For example,

$$ f(n) := 3^n $$

(Note: I may have made an error, and this may not actually be a valid sequence. It is clear, however, that more extreme examples, such as $f(n) := (6n)!$, are valid.)

The optimal strategy: Schelling fences

The trick to forming a strategy is to pick a desired payout at the beginning of the game and stick with it even though the expected payout for trying to pass over that payout will exceed the payout itself by the time you reach it.

This is (I believe--see below) called a Schelling Fence, and it's a concept with some wider implications. Unfortunately, I don't have time to flesh out the concept at the moment, let alone to start calculating an optimal Schelling Fence, so I will just leave some links for reference, and come back later to improve this section.

The Schelling Fence is based on the concept of a Schelling Point, which is actually quite different: as I understand it, Schelling Points are strategic choices or solutions that have more psychological value than game-theoretic value, and are therefore used to solve coordination problems when communication is impossible, even though there's not necessarily any strategic advantage to them within the game. Here's some background on Schelling Points:

http://lesswrong.com/lw/dc7/nash_equilibria_and_schelling_points/

(Wiki's post is shorter but less comprehensive; pick your poision: https://en.wikipedia.org/wiki/Focal_point_%28game_theory%29)

This concept is extended to the Schelling Fence in the following post. Unfortunately, I haven't yet figured out whether or not the concept is original to this post (and author); when I first read the post, I assumed the section on Schelling Fences was simply paraphrasing a known game-theoretic principle, but when I tried to research the principle further for the sake of this post, I found nothing.

http://lesswrong.com/lw/ase/schelling_fences_on_slippery_slopes/

  • Caught an arithmetic error. Will fix when I have time. – Kyle Strand Mar 07 '15 at 05:04
  • 1
    For an $n$-sided die you have $P_L(n) = \frac1n \left(1+\frac1n\right)^{n-1}$, which is approximately $\frac{e}{n+3/2}$. And $e^2\approx 7.389$, which is not too far from $6+\frac32$. So for $n=6$, $\frac{1}{P_L(n)} \approx 2.776$ which is as close to $e$ as this gets for any integer $n$. (For $n=7$ it's much closer to $\pi$.) So it is a coincidence that this coefficient "looks suspiciously like $e$", but not a coincidence that it looks even more like $\frac{7.5}{e}$. – David K Mar 07 '15 at 15:47
  • Hahahahaha. I don't think I'm ever going to get back to fixing the arithmetic error, especially since I no longer remember what it is. – Kyle Strand Jan 08 '16 at 17:35