Example of $\arg\min\limits_{T: E(T(X))=\theta}\mathbb{E}|T(X) - \theta| \neq \arg\min\limits_{E(T(X))=\theta}\mathbb{E}(T(X) - \theta)^2$

Question

Suppose that $X_1, X_2, \dots, X_n$ is an i.i.d. sample distributed according to $F(x, \theta)$ distribution function, and we want to estimate $\theta$.

Among all unbiased estimators, the best one is considered to be the one which has the minimum variance.

But what if we want to minimize not variance, but absolute deviation, e.g. find such statistics $T(X)$ which has minimum $\mathbb{E}|T(X_1, X_2, \dots, X_n) - \theta|$, not $\mathbb{E}(T(X_1, X_2, \dots, X_n) - \theta)^2$?

Can you provide examples of a distribution $F(x, \theta)$ and two unbiased estimators one of which has the least variance and another one – the least expected absolute deviation?

I tried to consider normal distribution. It is known that both the sample mean and the sample median are unbiased estimators for the mean. But I'm not sure that the median has the least expected absolute deviation here.

Any help, comments, hints and, especially, complete answers are very welcome and would be greatly appreciated!

Answer 1

The Laplace distribution seems like it would be the canonical example for this kind of thing. If $X$ is Laplace distributed with density:

\begin{equation} f_X(x) = \frac{1}{2b}\textrm{Exp}(-\frac{|x-\mu|}{b}) \end{equation}

The MLE for $\mu$ is the sample median $\hat{m}$, and the MLE for $b$ is the sample average absolute deviation from the median: $\hat{b} = \frac{1}{N}\sum_{n=1}^N |x_i - \hat{m}|$. Note that $\bar{x}$ is also unbiased for $\mu$ as the mean and median are the same in a Laplace distribution.

Denoting by $\mu_{m}$ and $\mu_{\mu}$ the sample median and sample mean, respectively, we can examine their variances and absolute deviations:

$E[(\mu_{\mu} - \mu)^2] = E[\mu_{\mu}^2 - 2\mu_{\mu}\mu + \mu^2] = E[\mu^2_{\mu}]-\mu^2$.

Expanding $\mu_{\mu}^2 = (\frac{1}{N}\sum_{i=1}^N x_i)^2 = \frac{1}{N^2}\sum_{i,j=1}^N x_ix_j$ Taking the expectation and noting the independence of the observations we get:

$E[\mu_\mu^2] = \frac{1}{N^2}(\sum_{i=1}^N E[x_i^2] + 2\sum_{i\neq j = 1}^N E[x_i]E[x_j] ) = \frac{1}{N^2}(2b^2 + \mu^2 + 2\binom{N}{2}\mu^2)$.

$E[\mu_\mu^2] = \frac{1}{N^2}(N2b^2 + N\mu^2 + 2\binom{N}{2}\mu^2) = \frac{1}{N^2}(N2b^2 + N\mu^2 + N*(N-1)\mu^2) = \frac{1}{N}(2b^2 + \mu^2 + (N-1)\mu^2)$

So $Var[\mu_\mu] = \frac{2b^2}{N}$.

I think you can do some similar stuff with the median, and also calculate the absolute deviation for $\mu_\mu$, but I ran out of time to do the algebra.