Let $X_1, \ldots , X_n$ be independent random variables having a common density with mean $\mu$ and variance $\sigma^2$.

Question

Let $X_1, \ldots , X_n$ be independent random variables having a common density with mean $\mu$ and variance $\sigma^2$. The sample variance is defined as the random variance of the sample:

$$S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2$$

where $\bar{X}=\frac1n\sum^n_{j=1}X_j$. Calculate $E[S^2]$.

Attempt

Is $X_i-\bar{X}=(X_i-\mu)-(\bar{X}-\mu)$? Assuming it is, then

$$\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\mu)-n(\bar{X}-\mu)$$

Now,

$$E\left[\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})\right]=E\left[ \frac{1}{n-1}\sum_{i=1}^{n}(X_i-\mu)-n(\bar{X}-\mu) \right]$$

I think you're missing a power of 2 in your formula for $S^2$ — , Oct 10 '21 at 00:46
$S^2_n$ is known to be an unbiased estimator of $\sigma^2$ under the assumptions of your problem. — Mittens, Oct 10 '21 at 00:48
@MatthewPilling In other words? I wrote the problem as is... — Max, Oct 10 '21 at 00:49
It follows from the definition of $\bar X$ that $\sum (X_i-\bar X)$ is just $0$. The sample variance is generally defined in terms of $(X_i-\bar X )^2$ rather than $X_i-\bar X$. Also, you can factor $\frac 1 {n-1}$ from the expectation, since it's a constant. — Acccumulation, Oct 10 '21 at 00:51
@Acccumulation What you are telling me is that $E(X_i-\mu)=Var(X_i)=\sigma^2$? — Max, Oct 10 '21 at 00:58
$$\begin{align} \sum^n_{j=1}(X_j-\bar{X})^2&=\sum^n_{j=1}X^2_j-n(\bar{X})^2=(1-\tfrac1n)\sum^n_{j=1}X^2_j-\frac2n\sum_{1\leq i<j\leq n}X_iX_j \end{align}$$ Taking expectations yields $$ E[S^2_n]=E[X^2_1]-\frac2{n(n-1)}\binom{n}{2}\mu^2=E[X^2_1]-\mu^2=\sigma^2$$ — Mittens, Oct 10 '21 at 01:10

score 1 · Accepted Answer · answered Oct 10 '21 at 01:05

Using the definition of variance:

$$ \begin{align} E\left(X^{2}\right)&=\sigma^{2}+\mu^{2}\\ E\left(\bar{X}^{2}\right)&=\frac{\sigma^{2}}{n}+\mu^{2} \end{align} $$

Then substitute these into the equation:

$$ \begin{align} E\left(S^{2}\right)&=E\left[\frac{1}{n-1}\sum_{i=1}^{n}{\left(X_{i}-\bar{X}\right)^{2}}\right]\\ \\ &= \frac{1}{n-1}E\left[\sum_{i=1}^{n}{X_{i}^{2}-n\bar{X}^{2}}\right]\\ \\ &=\frac{1}{n-1}\left[\sum_{i=1}^{n}{E\left(X_{i}^{2}\right)}-nE\left(\bar{X}^{2}\right)\right]\\ \\ &=\sigma^{2} \end{align} $$

score 1 · Answer 2 · answered Oct 10 '21 at 01:10

$$\begin{align} \sum^n_{j=1}(X_j-\bar{X})^2&=\sum^n_{j=1}X^2_j-n(\bar{X})^2=(1-\tfrac1n)\sum^n_{j=1}X^2_j-\frac2n\sum_{1\leq i<j\leq n}X_iX_j \end{align}$$ Taking expectations yields $$ E[S^2_n]=E[X^2_1]-\frac2{n(n-1)}\binom{n}{2}\mu^2=E[X^2_1]-\mu^2=\sigma^2$$

Let $X_1, \ldots , X_n$ be independent random variables having a common density with mean $\mu$ and variance $\sigma^2$.

2 Answers2