Expected number of distinct objects in sampling with replacement

Question

Given the set of numbers from 1 to n: { 1, 2, 3 .. n } We draw n numbers randomly (with uniform distribution) from this set (with replacement). What is the expected number of distinct values that we would draw?

My Approach:

Let $X(k)$ denote the expected number of distinct values in a sample of size $k$. Then,

$X(k) = \frac{n - X(k-1)}{n}*(1 + X(k-1)) + \frac{X(k-1)}{n}*X(k-1)$

$X(k) = 1 + \frac{n-1}{n}*X(k-1)$

Since $X(1) = 1$, solving the recursive relation, we get

$X(k) = 1 + (\frac{n-1}{n}) + (\frac{n-1}{n})^2 + (\frac{n-1}{n})^3 + ... + (\frac{n-1}{n})^{k-1}$

$X(k) = \frac{1-(\frac{n-1}{n})^k}{\frac{1}{n}} = n*(1-(1-\frac{1}{n})^k)$

Hence,

$X(n) = n*(1-(1-\frac{1}{n})^n)$

The answer is correct, but I doubt if my approach is correct or not. The idea behind the first equation is: after $k-1$th sample, the probability of getting a new value in $k$th sample is $ \frac{n - X(k-1)}{n} $. Since $X(k-1)$ is not necessarily an integer, I doubt if the probability is correct or not. So my question is: is my approach correct or not? please provide some convincing explanation as to why or why not is it correct.

Answer 1

The easier route is through using linearity of expectation.

Modifying the notation slightly, let $\{1, \dots, m\}$ denote the set composed of $m$ numbers and $n$ denote the number of draws we take from the set.

Let $Y_i$ be an indicator variable that is equal to $1$ if number $i \in \{1, \dots, m\}$ is not collected in $n$ draws, and $0$ otherwise.

Then $P(Y_i) = {(1- \frac{1}{m})}^n$

Now the expectation of an indicator variable is just the probability of the event it indicates, and by linearity of expectation, which operates even when the variables are not independent,

$\Bbb E[Y] = \Sigma \Bbb E[Y_i] = m(1-\frac{1}{m})^n$

and $\Bbb E[X]=$ expected number of distinct coupons collected in $n$ draws
= $m - \Bbb E[Y] = m[1- (1-\frac{1}{m})^n]$

Answer 2

In your answer $X(k)$ is defined to be the expectation of the number of distinct results if $k$ are drawn. That is wrong.

Let $X_k$ denote the number of distinct results if $k$ are drawn.

Then:$$\mathbb E[X_k|X_{k-1}=r]=\frac{r}{n}\cdot r+\left(1-\frac{r}{n}\right)\cdot(r+1)=\left(1-\frac1n\right)r+1$$ From this we conclude that:$$\mathbb E[X_k|X_{k-1}]=\left(1-\frac1n\right)X_{k-1}+1$$and consequently:$$\mathbb EX_k=\mathbb E[\mathbb E[X_k|X_{k-1}]]=\left(1-\frac1n\right)\mathbb EX_{k-1}+1\tag1$$

This can be further exploited as you do in your question, resulting in:$$\mathbb EX_k=n\left(1-\left(1-\frac1n\right)^k\right)$$ Substitution $k=n$ gives your final result.

---

Of course it is much more elegant to make use of linearity of expectation and symmetry (provided in the answer of @true blue anil). I am aware of it (now) that you already know that but IMV the method simply deserves to be mentioned in this context. Also for the benefit of persons who read this and are not familiar with it yet.

Answer 3

The idea is correct although it would be better if you can state the proof in a more rigorous way. In short, the reason why the current a-bit-not-rigorous proof works essentially is due to the linearity of the recurrence. More specifically, the $\frac{n - X(k-1)}{n}$ you wrote can be better written as $\frac{n - \mathbb{E}(X(k-1))}{n}$, which can be expanded to $\sum_x \mathbb{P}(X(k−1)=x) \frac{n - x}{n}$ without any problem since the form is linear.

---

Revised Proof

Let $X(k)$ be the random variable indicating the number of distinct values in a sample of size k.

Then we have the following recurrence: $$ X(k) = \frac{n - X(k-1)}{n}*(1 + X(k-1)) + \frac{X(k-1)}{n}*X(k-1) = 1 + \frac{n-1}{n}*X(k-1), $$ which gives \begin{align} \mathbb{E}(X(k)) & = \sum_x \mathbb{P}(X(k) = x) x \\ &= \sum_{x'} \mathbb{P}(X(k-1) = x') (1 + \frac{n-1}{n}*x') \\ &= 1 + \frac{n-1}{n}*\mathbb{E}(X(k-1)). \end{align} The above equations with and without expectation have the same form due to the linearity of the recurrence.

Now, starting from $\mathbb{E}(X(1)) = X(1)=1$, we solve the above recursive relation and get \begin{align} \mathbb{E}(X(k)) &= 1 + (\frac{n-1}{n}) + (\frac{n-1}{n})^2 + (\frac{n-1}{n})^3 + ... + (\frac{n-1}{n})^{k-1} \\ &= \frac{1-(\frac{n-1}{n})^k}{\frac{1}{n}} = n*(1-(1-\frac{1}{n})^k) \end{align}

Hence, $$ \mathbb{E}(X(n)) = n*(1-(1-\frac{1}{n})^n). $$

Answer 4

As @VezenBU suggested.

Let X(k) denote the number of distinct values in a sample of size k.

$\mathbb{E}[X(k)] = \sum_x P(X(k-1) = x) \{(\frac{n-x}{n})(1+x) + \frac{x}{n}x\} $

$\mathbb{E}[X(k)] = \sum_x P(X(k-1) = x) \{(1-\frac{x}{n})(1+x) + \frac{x^2}{n}\} $

$\mathbb{E}[X(k)] = \sum_x P(X(k-1) = x) \{1 - \frac{x}{n} + x\} $

$\mathbb{E}[X(k)] = \sum_x \{P(X(k-1) = x) - \frac{x}{n} * P(X(k-1) = x) + x *P(X(k-1) = x) \} $

$\mathbb{E}[X(k)] = 1 - \frac{\mathbb{E}[X(k-1)]}{n} + \mathbb{E}[X(k-1)]$

$\mathbb{E}[X(k)] = 1 + (\frac{n-1}{n})*\mathbb{E}[X(k-1)]$

$\mathbb{E}[X(k)] = n*(1-(1-\frac{1}{n})^n)$

Answer 5

This can also be done using Stirling numbers. We have from first principles for the expectation that it is

$$\frac{1}{n^n} \sum_{k=1}^n k {n\choose k} k! {n\brace k} \\ = \frac{1}{n^n} n! [z^n] \sum_{k=1}^n k {n\choose k} (\exp(z)-1)^k \\ = \frac{n}{n^n} n! [z^n] \sum_{k=1}^n {n-1\choose k-1} (\exp(z)-1)^k \\ = \frac{n}{n^n} n! [z^n] (\exp(z)-1) \sum_{k=0}^{n-1} {n-1\choose k} (\exp(z)-1)^k \\ = \frac{n}{n^n} n! [z^n] (\exp(z)-1) \exp((n-1)z) \\ = \frac{n}{n^n} n! [z^n] (\exp(nz)-\exp((n-1)z)) \\ = \frac{n}{n^n} (n^n - (n-1)^n) \\ = n \left(1-\left(1-\frac{1}{n}\right)^n\right).$$