1

Let $x_{1},...,x_{n} \sim F$ where the expected value of $F$ is $\mu$ and the variance is $\sigma^{2}$.

$ S_{n}^{2}=\frac{1}{n-1}\sum_{i=1}^{n}\left(x_{i}-\overline{x}\right)^{2} $

converges in probability to the variance. What I have tried so far is to use Chebyshev's in equality - since I know that $S_{n}$ is unbiased, given an $\epsilon >0$ we get:

$P\left(\left|S_{n}^{2}-\sigma^{2}\right|\geq\epsilon\right)\leq\frac{Var\left(S_{n}^{2}\right)}{\epsilon^{2}}$

I thought somehow with manipulation I can bound it with something that can converge to zero. However finding the variance of $S_{n}^{2}$ got really complicated, and I think that there should be an easier way. Should I proceed with finding the variance of $S_{n}^{2}$ ? Can somebody give me a clue? I would really appreciate it. Thanks!

mathstackuser123
  • 147
  • Possibly of interest: http://math.stackexchange.com/questions/72975/variance-of-sample-variance/73080#73080 –  Mar 17 '17 at 02:27

1 Answers1

3

$$S_{n}^{2}=\frac{1}{n-1}\sum_{i=1}^{n}\left(x_{i}-\overline{x}\right)^{2}=\dfrac{n}{n-1}\left({\overline{x^2}-\left(\overline x\right)^2}\right),$$ where $\overline{x^2} = \dfrac{\sum_{i=1}^n x_i^2}{n}$. Law of Large Numbers implies that $$ \overline{x^2} = \dfrac{\sum_{i=1}^n x_i^2}{n} \xrightarrow{p} \mathbb Ex_1^2=\sigma^2+\mu^2 \text{ as } n\to\infty $$ and $$ \overline{x} = \dfrac{\sum_{i=1}^n x_i}{n} \xrightarrow{p} \mathbb Ex_1=\mu \text{ as } n\to\infty. $$ And $\tfrac{n}{n-1}\to 1$ as $n\to\infty$. By the properties of convergence in probability, $$ S_{n}^{2}= \underbrace{\dfrac{n}{n-1}}_{\begin{matrix}\downarrow\\ 1\end{matrix}}\biggl({\underbrace{\overline{x^2}}_{\begin{matrix}\downarrow_p \\ \sigma^2+\mu^2\end{matrix}}-\underbrace{\left(\overline x\right)^2}_{\begin{matrix}\downarrow_p \\ \mu^2\end{matrix}}}\biggr) \xrightarrow{p} 1\cdot (\sigma^2+\mu^2 - \mu^2)=\sigma^2. $$

You cannot use Chebyshev inequality without any knowledge on higher moments of samples. The variance $Var(S_n^2)$ exists if 4th moment exists: $$\mathbb Ex_1^4<\infty.$$ And we are given only that first two moments exist, and this is sufficient for convergence in probability of $S_n^2$.

NCh
  • 14,089
  • 4
  • 18
  • 33
  • Why can't we use Chebyshev without knowledge of higher moments? – lioumens Mar 17 '17 at 02:47
  • @RationalHusky The answer is above: variance $Var(S^2_n)$ exists if (and only if) $4$th moment exists: – NCh Mar 17 '17 at 12:15
  • Thanks for the feedback! @NCh - So the above statement is true only when we know that the fourth moment exists? Because while calculating the variance I saw that I had to know that the fourth moment exists. Even in the assumption that it exists I couln't bound it by something that converges to zero. – mathstackuser123 Mar 20 '17 at 09:14
  • @NCh by the way, something that gets me confused a lot. If I understood correctly what you did above, you treated the mean $ \overline{x}$ as a constant and not as a random variable, why is that? They do this a lot in the statistics courses that I took\taking. Sometime we treat it as random variable and sometimes as constant (mean of the sample) – mathstackuser123 Mar 20 '17 at 09:17
  • @mathstackuser123 1) I never treat $\overline x$ as a constant. 2) the above statement on convergence in probability is valid under the condition that the variance $Var(x_1)$ exists only. This convergence is a simple consequence of weak law of large numbers: https://en.wikipedia.org/wiki/Law_of_large_numbers#Weak_law. But you tried to prove this convergence using Chebyshev inequality. You cannot use this instrument when 4th moments are infinite. In this case right hand side of inequality is infinite too. – NCh Mar 20 '17 at 10:56