5

A student asked me this probability question today.

Suppose that a Bernoulli trial with probability of success $\frac{1}{n}$ is repeated $n$ times. What is the probability of no successes?

A simple counting argument shows the answer is $P_n=\frac{(n-1)^n}{n^n}=(1-\frac{1}{n})^n$. Geometrically we are looking at a ratio of volumes of two $n$-dimensional hypercubes.

Soon afterward it occured to me that $\lim_{n\to\infty} P_n=\frac{1}{e}$.

My question is whether this limit can be interpreted as a probability in any sense, or whether this is just a geometric fact about ratios of volumes of hypercubes. The difficulty is that the sample space is changing as $n\to\infty$. Each term is the probability of an event from a uniform distribution on a finite set, but in the limit the underlying set seems to be $\mathbb{N}^\mathbb{N}$ and I have no idea what the distribution should be.

Any ideas?

beanstalk
  • 675

1 Answers1

8

Yes, it can. You can ask more generally for the probability of $k$ successes for any $k$, so we're looking at a sequence of binomial distributions $X_n \sim B(n, \frac{1}{n})$. We have

$$\mathbb{P}(X_n = k) = {n \choose k} \frac{1}{n^k} \left( 1 - \frac{1}{n} \right)^{n-k}$$

and as $n \to \infty$ this converges to $e^{-1} \frac{1}{k!}$. These limits themselves are non-negative reals adding up to $1$ so also describe some distribution, namely the Poisson distribution $X \sim \text{Pois}(1)$ with parameter $1$; our calculations are saying that the sequence of binomial distributions $X_n$ converges to $X$ (more or less). This is an important special case of the Poisson limit theorem, and is "where the Poisson distribution comes from." In particular $e^{-1}$ is the probability $\mathbb{P}(X = 0)$.

The sample space is just $\mathbb{N}$. Of course when we pass to the binomial distribution we flatten the picture of the situation; we pass from a bunch of uniform distributions to Bernoulli distributions to their sum. Your comment about the sample space looking like $\mathbb{N}^{\mathbb{N}}$ in the limit can be made precise as follows: the process of taking $n$ random samples from the uniform distribution on $\{ 1, 2, \dots n \}$ itself also has a limit in a suitable sense, namely a (discrete) Poisson point process. The number of samples contained in any subset

$$S \subseteq \{ 1, 2, \dots n \}$$

of size $s$ is a binomial distribution $B(n, \frac{s}{n})$, and as $n \to \infty$ (keeping $s$ fixed) this has a limit $\text{Pois}(s)$. So we can think about the limit process as a process of "choosing infinitely many random points from $\mathbb{N}$," in such a way that the distribution of the number of points in any subset $S \subset \mathbb{N}$ of size $s$ is $\text{Pois}(s)$. This works out to be equivalent to saying that the number of points equal to any $n \in \mathbb{N}$ is $\text{Pois}(1)$, and each of these Poisson distributions is independent. Surprisingly we can make sense of this even though there is no uniform distribution on $\mathbb{N}$! (Note however that we are only counting points, so for us the points have become indistinguishable; there is no "first point," "second point," etc. so no marginal distribution over the locations of such points, which would have to be the nonexistent uniform distribution on $\mathbb{N}$.)

A continuous variant of this limiting construction is to consider taking $n$ random samples from the uniform distribution on $[0, n]$. We can recover the finite sample spaces above by rounding up. Here the analogue of the computation above is that the number of samples in any interval of length $\lambda$ is $B(n, \frac{\lambda}{n})$, and as $n \to \infty$ this converges to $\text{Pois}(\lambda)$. The limit is a Poisson point process on $[0, \infty)$, "choosing infinitely many random points from $[0, \infty)$," where the number of points in any interval of length $\lambda$ is $\text{Pois}(\lambda)$. This is used to model, say, rain, where the random points correspond to raindrops.

Qiaochu Yuan
  • 468,795
  • Thanks! Re: your comment about interpreting this as a distribution on $\mathbb{N}^\mathbb{N}$ - Since the points have become indistinguishable, is it accurate to think of Pois(s) as a distribution on $\mathbb{N}^\mathbb{N}$ mod the action of the symmetric group $S_\mathbb{N}$ on the copy of $\mathbb{N}$ in the exponent? – beanstalk Jan 23 '25 at 23:21
  • @beanstalk: yes, that's one way to think about it (although $\text{Pois}(s)$ itself is just a random variable valued in $\mathbb{N}$ - the entire Poisson process is a larger object). – Qiaochu Yuan Jan 23 '25 at 23:32