4

Let $X_1, X_2, \ldots, X_n$ be i.i.d. with a common density function:

$$f(x;\theta)=\theta x^{\theta-1}$$ for $0<x<1$ and $\theta>0$. So this is $\operatorname{BETA}(\theta,1)$ distribution.

Given this Maximum Likelihood Estimator (MLE) for $\theta$: $$\hat \theta=\frac{-n}{\sum_{i=1}^n \ln(X_i)}$$

Determine if $\hat\theta$ is biased. If it is biased, could you redefine it to make it unbiased?

Unfortunately, this is where I get stuck; I have no idea how to evaluate the expectation of $\hat\theta$. $$\operatorname E\left( \frac{-n}{\sum_{i=1}^n \ln(X_i)} \right)$$

How does one calculate this?

StubbornAtom
  • 17,932
EllipticalInitial
  • 2,533
  • Very closely related to this recent Question. Not voting to say it's a duplicate, but you might get something useful from the Comments there. Short version: $\hat \theta$ is biased. – BruceET Feb 15 '18 at 03:03
  • I suspected that, but the issue is that I'm not really sure how to mathematically show that it is. – EllipticalInitial Feb 15 '18 at 03:06
  • 1
    Nor am I. I have used simulation to convince myself that neither MLE or MOM is unbiased. Wikipedia is not much help on unbiasedness. Haven't looked at Bayes'. – BruceET Feb 15 '18 at 03:22
  • 2
    It is similar to this https://math.stackexchange.com/questions/2619937/expected-value-for-a-combination-of-iid-from-fx-theta-frac-theta-x and the solution is almost the same! – Maffred Feb 15 '18 at 03:46
  • If you have any question on the question I linked to I can expand here. – Maffred Feb 15 '18 at 03:58
  • I tried it, but I ended up with some unusual distribution for the $Y=ln(X_i)$. I ended up with a PDF looking like this: $$f_Y(y)=\theta e^{y\theta}$$ where $-\infty<y<0$. Not sure where to go with that. – EllipticalInitial Feb 15 '18 at 04:06
  • 1
    If you just want to show biasness apply Jensen and calculate $E(log x) = -1/\theta$. – Maffred Feb 15 '18 at 04:15
  • I attempted that but ended up with an inconclusive result since all it ends up showing is that the expected value of $\hat\theta$ is less than or equal to the parameter we're trying to estimate. – EllipticalInitial Feb 15 '18 at 04:44
  • 1
    $E(\hat\theta) > \theta$ because $f(t)= \frac 1 t$ is strictly convex. Thus it is biased. – Maffred Feb 15 '18 at 04:53
  • Oh, I see it now. Thank you very much. But now there's that second part of the question. – EllipticalInitial Feb 15 '18 at 05:08
  • Calculate the bias and subtract it. To calculate it you first calculate the distribution of logx, then you calculate the distribution of the sum of two logs with convolution and use then induction for $n$. You get the distribution of $log x_1 + log x_2 + \cdots +log x_n$. When you have the distribution of $Y$ you can compute the distribution of $\frac 1 Y$ using $\mathbb P(\frac 1 Y \leq t) = \mathbb P(Y \geq \frac 1 t) = 1 - \mathbb P(Y < \frac 1 t) = 1- F(\frac 1 t) $ then derive. This can be done for $Y,t >0$. – Maffred Feb 15 '18 at 05:49
  • This is what I think. I'm not an expert in probability and statistics. You could create a new question to ask for $\operatorname E\left( \frac{-n}{\sum_{i=1}^n \ln(X_i)} \right)$ and see what experts say. – Maffred Feb 15 '18 at 05:53

1 Answers1

3

Define, $$ Y = -\ln X, $$ thus $$ F_Y(y) = \mathbb{P}(Y\le y)=\mathbb{P}(X\ge e^{-y}) = 1-F_X(e^{-y}), $$ hence, $$ f_X(x) = f_X(e^{-y})e^{-y}=\theta e^{-y (\theta - 1)}e^{-y} = \theta e^{-y\theta}, \quad y>0. $$ Therefore, $$ - \sum_{i=1}^n\ln X_i \sim \mathcal{G}amma(n,\theta). $$ As such, $$ \frac{n}{-\sum_{i=1}^n \ln X_i} \sim n\times \mathcal{I}nv\mathcal{G}amma(n, \theta), $$ then the expectation is $$ \mathbb{E}[\hat{\theta}_n] = \frac{n}{n-1}\theta > \theta, $$ namely, $$ \tilde{\theta}_n = \frac{n-1}{n}\hat{\theta}_n, $$ is an unbiased estimator of $\theta$.

V. Vancak
  • 16,927