Let $X_1, \ldots, X_n$ be a random sample of the normal distribution with parameters $(\theta, \theta^2)$. How can I find a sufficient statistic for $\theta$?
1 Answers
Let $\bar x = (x_1+\cdots+x_n)/n$ and recall from algebra that $$ \sum_{i=1}^n (x_i-\theta)^2 = n(\bar x-\theta)^2 + \sum_{i=1}^n (x_i-\bar x)^2. $$
Then deal with the density: \begin{align} & f_{X_1,\ldots,X_n}(x_1,\ldots,x_n) \\[10pt] \propto {} & \prod_{i=1}^n\frac 1 \theta \exp\left( \frac{-1} 2 \left( \frac{x_i-\theta}{\theta} \right)^2 \right) \\[10pt] = {} & \frac 1 {\theta^n} \exp\left( \frac{-1}{2\theta^2} \sum_{i=1}^n (x_i-\theta)^2 \right) \\[10pt] = {} & \theta^{-n} \exp\left( \frac{-1}{2\theta^2} \left( n(\bar x - \theta)^2 + \sum_{i=1}^n (x_i - \bar x)^2 \right) \right) \\[10pt] = {} & \theta^{-n} \exp\left( - \frac n {2\theta^2} ((\bar x - \theta)^2 + s^2) \right). \end{align} That the density depends on $(x_1,\ldots,x_n)$ only through $\bar x$ and $s^2$ is enough to conclude that that pair is sufficient. (If there had also been an additional factor depending on $(x_1,\ldots,x_n)$ through other functions but not depending on $\theta$, that would also be enough. That happens, e.g., with the Poisson distribution. This is Fisher's factorization criterion.)
-
Shouldn't the $s^2$ be $\frac{1}{n-1}\sum_{i=1}^{n}(x_i - \bar{x})^2$ instead of $\frac{1}{n}\sum_{i=1}^{n}(x_i - \bar{x})^2$? And I have a follow-up question: is $(\bar{x}, s^2)$ also minimal sufficient? – Bayequentist Mar 18 '20 at 06:50
-
1@Bayequentist : Usually in beginning textbooks, and in many other contexts, $s^2$ is defined as $\frac 1 {n-1} \sum_{i=1}^n (x_i - \overline x)^2.$ That is the best unbiased estimator of $\sigma^2$ for the family of normal distributions. The MLE is $\frac 1 n \sum_{i=1}^n (x_i -\overline x)^2,$ and in some contexts the notation $s^2$ may refer to that. But either way, it depends on $(x_1,\ldots,x_n)$ only through $\sum_{i=1}^n (x_i - \overline x)^2.$ That pair is also a minimal sufficient statistic. I wonder whether the question of how to prove that has been posted here. – Michael Hardy Mar 19 '20 at 19:58