Polya's urn, should I use martingales or LLN

Question

I am trying to prove the following question, but I am finding it a bit tricky to determine the distribution of $X_i$ (the number of red balls drawn in the $i$-th round) and thus I do not know which approach I should use to prove this. All help is appreciated!

Consider an urn with r red and b blue balls initially present. In each round a ball is drawn from the urn uniformly at random, and then replaced together with another ball of the same colour. Let $R_n$ denote the number of red balls in the urn after n rounds. Show that, almost surely, $R_n \to \infty$ as $n \to \infty$, i.e. that red balls are drawn infinitely many times.

My first thought is the following: Since after each round the ball is replaced together with an identical copy, we should have $(r+b)+i$ balls in the urn after $i$ rounds and therefore the number of balls is constantly growing for each round, and each of these $(r+b)+i$ balls is equally likely to be drawn. Because of this, the variables in the sequence are not independent and the distribution depends on the outcome of the previous round, so, $X_i$ should be Bernoulli distributed with parameter p. And if this is right, I should not be able to use the law of large numbers to prove this since the Law of Large Numbers typically assumes that the random variables are identically and independently distributed. If this is true, I should probably use Martingales to prove the question instead.

However, when I looked around with some classmates, chatGPT and read on the internet, it still seems that some say that $X_i$ in this case is Binomial distributed with parameter $p$ and goes on to use the law of large numbers to prove this.

Since this type of urn-problem is new to me I do not feel certain with which way is the correct way to proceed proving this (Martingales or LLN) and I would appreciate all the help to clarify which way is correct and what the distribution of $X_i$ is in this case!

Answer 1

Let $r,b$ the number of red and blue balls which we have initially. Define $R_n=r+S_n$ where $S_n=X_1+...+X_n$ where $X_k$ is $1$ if red ball is drawn at turn $k$, and $0$ otherwise; set $S_0=0$. If you define $$\Theta_n=\frac{R_n}{r+b+n}$$ then this represents the proportion of red balls at the end of turn $n$. Consider the filtration given by the sequence of $\sigma$-algebras $\mathscr{F}_n:=\sigma(X_k,k\leq n)$. One may then notice that $E[X_{n+1}|\mathscr{F}_n]=\Theta_n$. So we proceed to show that $(\Theta_n)_{n \in \mathbb{N}}$ is a $\mathscr{F}_n$-martingale: it is uniformly bounded in $[0,1]$, so that $\Theta_n \in L^1,\,\forall n$ and $$E[\Theta_{n+1}|\mathscr{F}_n]=\frac{R_n+\Theta_n}{r+b+n+1}=\frac{(r+b+n)\Theta_n+\Theta_n}{r+b+n+1}=\Theta_n$$ We also note that $(\Theta_n)_{n \in \mathbb{N}}$ is uniformly integrable, so that $\Theta_n\to \Theta$ a.s. for some $\Theta \in L^1$ by martingale convergence. So for a.a. $\omega \in \Omega$ we have $$ R_n(\omega)=(r+b+n)\cdot \frac{R_n(\omega)}{r+b+n}\to \infty \cdot \Theta(\omega) $$ now to conclude we need $\Theta>0$ a.s. This follows from the fact that $\Theta\sim \textrm{Beta}$, which has a continuous Lebesgue density, so that $P(\Theta=0)=0$.