5

I need to prove that using maximum likelihood estimation on both parameters of normal distribution indeed maximises likelihood function.

So, the log-likelihood function for parameters $\sigma$ and $m$ is $$ \ln L = -\frac{n}{2}\ln2 \pi - n \ln \sigma - \sum_{i=1}^n\dfrac{1}{2\sigma^2}(x_i - m)^2 $$

After differentiating we get two equations $$ \dfrac{\partial \ln L}{\partial \sigma} = - \dfrac{n}{\sigma} + \sum_{i=1}^n\dfrac{1}{\sigma^3}(x_i - m)^2 = 0$$ $$ \dfrac{\partial \ln L}{\partial m} = \sum_{i=1}^n\dfrac{1}{\sigma^2}(x_i - m) = 0 $$

And now we get the estimators: $$ m = \dfrac{\sum_{i=1}^n x_i}{n} $$ $$ \sigma = \sqrt{\dfrac{\sum_{i=1}^n\left(x_i - \dfrac{\sum_{i=1}^n x_i}{n}\right)^2}{n}}$$

Now I need to prove that this is local maximum. To do this I need to get second-order derivatives, and check that Hessian matrix is negative-definite.

The derivatives are $$ A = \dfrac{\partial^2 \ln L}{\partial \sigma^2} = \dfrac{n}{\sigma^2} - \sum_{i=1}^n\dfrac{3}{\sigma^4}(x_i - m)^2 = \dfrac{-2n^2}{\sum_{i=1}^n\left(x_i - \dfrac{\sum_{i=1}^n x_i}{n}\right)^2} $$ $$ C = \dfrac{\partial^2 \ln L}{\partial m^2} = -\dfrac{n}{\sigma^2} = -\dfrac{n^2}{\sum_{i=1}^n\left(x_i - \dfrac{\sum_{i=1}^n x_i}{n}\right)^2} $$ $$ B^2 = \left(\dfrac{\partial^2 \ln L}{\partial m \partial \sigma}\right)^2 = \left(\dfrac{-2\sum_{i=1}^n (x_i-m)}{\sigma^3}\right)^2 = \dfrac{4\left[\sum_{i=1}^n\left(x_i - \dfrac{\sum_{i=1}^n x_i}{n}\right)\right]^2}{\left(\sum_{i=1}^n(x_i - m)^2\right)^3} $$

And that's where I get lost. I should prove that $AC - B^2 > 0$ but it doesn't look to me as something clearly positive.

Algar
  • 53
  • Just a quick comment on terminology: when you are dealing with $x_i$, your functions are termed estimates, whereas if you work with the random quantities $X_i$, the functions are called estimators. – lokodiz Sep 05 '13 at 11:46
  • 1
    Also, it is not true that, for example, $m = \frac{\sum_{i=1}^n x_i}{n}$; rather, people commonly right $\hat{m} = \frac{\sum_{i=1}^n x_i}{n}$. Additionally, you should probably make it more explicit that you are evaluating the Hessian at the MLEs. – lokodiz Sep 05 '13 at 11:52
  • @lokodiz how would you prove that this local maximum is an absolute maximum ? – Gabriel Romon Apr 14 '17 at 16:29
  • Your statement Now I need to prove that this is local maximum. is not correct. You need to prove that the point is the global maximum. The log-likelihood function $\log l(\mu,\sigma^2)$ is not jointly concave in both the mean $\mu$ and variance $\sigma^2$, but it is concave in each of them; see here for more details. – Amir Mar 11 '25 at 16:29
  • To handle the maximization procedure, first maximize $\log l(\mu,\sigma^2)$ with respect to $\mu$ (the maximizer is the sample mean $\bar{x}$, and does not depend on $\sigma^2$). Then, replace the mean by the sample mean in the log-likelihood function, and maximize the resulting function $\log l(\bar{x},\sigma^2)$ with respect to the variance $\sigma^2$. Note the replacement does ruin the concavity of $\log l(\bar{x},\sigma^2)$. – Amir Mar 11 '25 at 16:33

1 Answers1

3

Set $\frac{\sum_{i=1}^{n}x_i}{n}=\bar{x}$. Then the numerator of $B$ is: $$\sum_{i=1}^n x_i - \bar{x} = \left(\sum_{i=1}^n x_i \right)- n\bar{x} = n\bar{x} - n\bar{x} = 0$$ Thus, $B=0$, and since $AC > 0$, we are done.

Starlight
  • 2,684
lokodiz
  • 2,350