Proof of the unbiasedness of an estimate obtained by gradient search

Question

As the solution to an MLE problem cannot be obtained analytically, I apply gradient search to find the solution $\hat{\theta}$ to satisfies \begin{align} \frac{\partial \mathcal{L}(\mathbf{y}\mid\theta)}{\partial \theta} = 0 \end{align} Without the expression of $\hat{\theta}$, how can I prove that \begin{align} \mathbb{E}(\hat{\theta}) = \theta \end{align} Are there any standard methods?

Answer 1

the question appears vague...so also the answer cannot be at top

by the way, in some situation, the Bartlett Identity can helps

First Bartlett Identity: $\mathbb{E}[\frac{\partial}{\partial\theta}l(\theta;\mathbf{x})]=0$

In fact can happens that in the expression of the score you can find the expression of $\hat{\theta}$ so, using the above mentioned identity perhaps you can derive the expected value you are looking for

If you are interested in, I can think at a particular exercise to better explain what I am saying.

EXAMPLE, as requested

Let's have a random sample $(X_1,...,X_n)$ from the following density, $\theta, x >0$

$f(x;\theta)=\sqrt{\frac{\theta}{2\pi}}x^{-\frac{3}{2}}e^{\frac{-\theta}{2}\frac{(x-1)^2}{x}}$

Suppose to have the following estimator for $\frac{1}{\theta}$:

$T=\frac{1}{n}\sum_i\frac{(x_i-1)^2}{x_i}$

Tell if T is unbiased for $\frac{1}{\theta}$ that's mean: prove that

$\mathbb{E}[T]=\frac{1}{\theta}$

SOLUTION

1) calculate the likeihood

2) take the log

3) derive with respect to $\theta$ (this is the score) finding

$Score(\theta)=\frac{n}{2\theta}-\frac{nT}{2}$

Applying the first Bartlett identity you get

$\mathbb{E}[\frac{n}{2\theta}-\frac{nT}{2}]=0$

That's mean

$\frac{n}{2\theta}=\frac{n}{2}\mathbb{E}[T]$

that is also

$\mathbb{E}[T]=\frac{1}{\theta}$