7

For the last few months, I have been interested in learning more about the Polya Urn Problem (https://en.wikipedia.org/wiki/P%C3%B3lya_urn_model):

Consider an urn containing $b$ blue balls and $r$ red balls initially. At each step:

  1. Draw a ball randomly from the urn.
  2. Return the drawn ball to the urn.
  3. Add $\alpha$ balls of the same color as the drawn ball to the urn.

Let $X_n$ be the proportion of blue balls after $n$ draws. Then:

$$X_n = \frac{B_n}{B_n + R_n}$$

Where $B_n$ and $R_n$ are the numbers of blue and red balls after $n$ draws, respectively. As $n$ approaches infinity, the distribution of $X_n$ converges to a beta distribution:

$$X_\infty \sim \text{Beta}(\frac{b}{\alpha}, \frac{r}{\alpha})$$

I am very interested in learning more about how to show that the ratio of colors converges to a Beta Distribution.

In earlier questions, I was able to show that the ratio of colors can be modelled as a Martingale (Probability of Drawing Balls from a Hat with Changing Number of Balls)

The more I read about this question, it seems to me that Martingales are not immediately necessary in proving the limiting distribution of the Polya Urn. It just seems like a side fact that the ratio of colors is Martingale, but we don't need to rely on this fact to prove the limiting distribution follows a Beta Distribution. Proving that the limiting distribution follows a Beta Distribution can be done by other techniques in math.

Can someone please help me understand why Martingales are immediately important in proving that the limiting distribution of Polya's Urn follows a Beta Distribution?

References:

RobPratt
  • 50,938
konofoso
  • 681
  • 1
    I think you can prove this using de Finetti's theorem, without using Martingales I suppose. – Mathieu Rundström Sep 28 '24 at 05:19
  • It is not strictly necessary to use martingales, one way of identifying the limiting distribution is by doing a moment computation. Having the martingales nevertheless helps because you a priori know that $X_n$ converges by the martingale convergence theorem and because $\vert X_n \vert \leq 1$ also its moments have to converge. – David Oct 01 '24 at 10:39
  • That is also actually the way Eggenberger and Polýa showed the convergence, by computing factorial moments before the idea of martingales was developed: https://onlinelibrary.wiley.com/doi/abs/10.1002/zamm.19230030407 – David Oct 01 '24 at 10:50
  • Similar to de Finetti's theorem, they exploit the exchangeability of Polýa's urn – David Oct 01 '24 at 10:51

1 Answers1

5

You don't need to use the concept of martingale to prove that the proportion of blue balls in the urn (hereafter PBBITU) follows the beta distribution. The similarity is in the formulas used in both proofs.

Next, as we will see later, we don't need $r$ and $b$ to be integers; being positive is enough. Thus, we divide both by $\alpha$: $$r' := \frac{r}{\alpha}; \; b' := \frac{b}{\alpha}$$ and assume that we initially have $r'$ red balls and $b'$ blue balls and that we add one ball at a time. This scaling won't affect the PBBITU.

If you plot the dynamics of the number of balls of each colour, you get a recombining binomial tree, as in the picture below.

enter image description here

We can go up (draw a red ball) or down (draw a blue ball) at each node. Thus, if you have $T$ "forks", you have $2^T$ distinct paths to a node at time $T$. The plot shows two paths to $(b'+2, r'+3)$, rrrbb and bbrrr; the total number of paths to $(b'+2, r'+3)$ is $\binom{5}{2} = \frac{5!}{2!(5-2)!} = 10$. In the general case, there are $\binom{T}{B} = \frac{T!}{B!(T-B)!}$ paths with $B$ blue balls and $T-R$ red balls.

The probability of following a path that leads to $b'+B, r'+(T-B)$ is $$\frac{\prod_{j=0}^{B-1}(b'+j)\prod_{i=0}^{T-B-1}(r'+i)}{\prod_{k=0}^{T-1}(b'+r'+k)}$$ Let me know if you need me to write it explicitly.

Thus, the (overall) probability to get to $b'+B, r'+(T-B)$ at $T$ is

$$p_T\left(b'+B, r'+(T-B)\right)=\underbrace{\binom{T}{B}}_{\text{How many paths lead to }b'+B, r'+(T-B)} \underbrace{\frac{\prod_{j=0}^{B-1}(b'+j)\prod_{i=0}^{T-B-1}(r'+i)}{\prod_{k=0}^{T-1}(b'+r'+k)}}_{\text{The probability to follow any of these }\binom{T}{B}\text{ paths}}=\star$$

Now, what's left is to prove that the distribution with this density converges to beta distribution. The proof of this convergence will be similar to Polya's urn model - limit distribution. If we use $\Gamma(s+1) = s\Gamma(s)$ property of Gamma function

\begin{align} \Gamma(b'+B)&=\prod_{j=0}^{B-1}(b'+j)\Gamma(b')\\ \Gamma(r'+T-B)&=\prod_{i=0}^{T-B-1}(r'+i)\Gamma(r')\\ \Gamma(b'+r'+T)&=\prod_{k=0}^{T-1}(b'+r'+k)\Gamma(b'+r') \end{align}

and develop the binomial coefficient, we obtain

\begin{align} \star&=\frac{T!}{B!(T-B)!} \frac{\Gamma(b'+B)\Gamma(r'+T-B)\Gamma(b'+r')}{\Gamma(b')\Gamma(r')\Gamma(b'+r'+T)}\\ &=\frac{\Gamma(b'+r')}{\Gamma(b')\Gamma(r')} \frac{\Gamma(b'+B)/B! \; \Gamma(r'+T-B)/(T-B)!}{\Gamma(b'+r'+T)/T!}=\spadesuit\\ \end{align}

Next, we use the Beta function and introduce $E_n(z) = \frac{\Gamma(n+z)}{n!n^{z-1}}$

\begin{align} \spadesuit&=\frac{1}{B(b', r')} \frac{E_B(b') B^{b'-1} \; E_{T-B}(r') (T-B)^{r'-1} }{E_{T}(b'+r') T^{b'+r'-1} }\\ &=\frac{1}{B(b', r')} \frac{1}{T} \left(\frac{B}{T}\right)^ {b'-1} \left(1-\frac{B}{T}\right)^ {r'-1}\frac{E_B(b') E_{T-B}(r')}{E_{T}(b'+r')} \end{align}

Now, to prove that the discrete distribution with these probabilities converges to a beta distribution (see this question for the rigorous definition and proof), let's compute the moment-generating function:

\begin{align} \mathbb{E}\left[e^{\lambda X_T}\right]&= \frac{1}{B(b', r')} \frac{1}{T} \sum_{B=0}^T e^{\lambda \frac{b'+B}{b'+r'+T}} \left(\frac{B}{T}\right)^ {b'-1} \left(1-\frac{B}{T}\right)^ {r'-1}\frac{E_B(b') E_{T-B}(r')}{E_{T}(b'+r')} \end{align}

We need to prove that this function converges to the moment-generating function of beta distribution:

$$\mathbb{E}\left[e^{\lambda X_\infty}\right] ?= \frac{1}{B(b', r')} \int_{0}^{1} e^{\lambda p} p^{b'-1}(1 - p)^{r'-1} dp$$

The last-but-one is similar to the Riemann sum of the integral just above $\left(\frac{B}{T} \text{ becomes } p \text{ and } \frac{1}{T} \text{ becomes } dp \right)$.

Since we integrate over a finite interval, it's sufficient to prove that

\begin{align*} e^{\lambda \frac{B}{T}} \xrightarrow[T \to \infty]{} e^{\lambda \frac{b'+B}{b'+r'+T}}\\ \frac{E_B(b') E_{T-B}(r')}{E_{T}(b'+r')} \xrightarrow[T-B, B \to \infty]{} 1\\ \end{align*}

Taylor expansion of expanentials can prove the former. The latter can be demonstrated using Stirling's approximation and Stirling's formula of the Gamma function: \begin{align*}\frac{n!}{\sqrt{2\pi n}\frac{n^n}{e^n} } \xrightarrow[n \to \infty]{} 1 \text{ and } \frac{\Gamma(x+1)}{\sqrt{2\pi x}\left(\frac{x}{e}\right)^x} \xrightarrow[x \to \infty]{} 1 \\ \strut \\ \end{align*}

Therefore, for $E_n(z) = \frac{\Gamma(n+z)}{n!n^{z-1}}$ we have $$\frac{E_n(z) \sqrt{2\pi n}\frac{n^n}{e^n}}{\sqrt{2\pi (n+z-1)}\left(\frac{n+z-1}{e}\right)^{n+z-1}} n^{z-1} = \frac{\Gamma(n+z)\sqrt{2\pi n}\frac{n^n}{e^n}}{n!\sqrt{2\pi (n+z-1)}\left(\frac{n+z-1}{e}\right)^{n+z-1}} \xrightarrow[n \to \infty]{} 1$$

Rearranging the terms of the first expression results in

$$\frac{E_n(z) {e^{z-1}}}{\left(\frac{n+z-1}{n}\right)^{n+z-0.5}} = \frac{E_n(z) {e^{z-1}}}{\left(\left(\frac{n+z-1}{n}\right)^{\frac{n+z-0.5}{z-1}}\right)^{z-1}} \xrightarrow[n \to \infty]{} 1$$

Factoring in the definition of $e$, we conclude that $E_n(z) \xrightarrow[n \to \infty]{} 1$. Therefore, the latter statement is correct.

Yulia V
  • 1,450
  • 1
    Wow! Thank you so much! Spasibo bolshoe)))) – konofoso Oct 05 '24 at 04:48
  • 1
    You have done such a wonderful job explaining these concepts to me ... I would really appreciate if you could write this proof about "distribution with this density converges to beta distribution." thank you so much Yulia V.... – konofoso Oct 05 '24 at 04:50
  • 1
    @konofoso - sure, will do later today – Yulia V Oct 05 '24 at 10:51
  • thank you so much! – konofoso Oct 06 '24 at 06:08
  • BTW - I checked out your blog that you posted on your stackoverflow profile (the link http://www.yu51a5.org/ was broken so I searched it on google (https://yu51a.org/) and it took me to the new link) - it is really interesting! :) – konofoso Oct 07 '24 at 03:21
  • Another question - how did you make this beautiful colored tree diagram? Did you use Python for this? – konofoso Oct 09 '24 at 02:44
  • @konofoso : all done (I think). Yes, I used Python, you can find the code here -> https://github.com/yu51a5/SE_snippets/blob/main/polya_urn_problem.py – Yulia V Oct 11 '24 at 02:45
  • Hi Yulia, I hope everything is going well with you. I was revisiting this answer and had a question. At some point, do you need to show that the ratio is martingale? I know that in Polya Urn, we usually show that the expected ratio at infinite time is equal to the expected ratio at some finite time n as n grows large. Did you need to rely on this fact during the proof? E.g. we argue using martingales that the urn has some stable behavior - and then derive the probability distribution of this stable behavior using this math that you have shown? – konofoso Nov 23 '24 at 07:18