7

In the preliminary and notation part of Andrzej Cichocki et al.'s paper, the authors provide some identities, e.g., for any symmetric positive definite matrix $\bf P$, the following holds

$$\operatorname{tr} ({\bf P}) - \log \det({\bf P}) \geq n \tag{14}$$

I was wondering how one would prove the above identity. I have read about the identity in Joe Whittaker's Graphical Models in Applied Multivariate Statistics (1st edition), but there it is only written that the identity is a corollary of a special case of Kullback-Leibler divergence. So, how can we prove the above identity using the divergence?

Rodrigo de Azevedo
  • 23,223
Frank wiene
  • 71
  • 1
  • 2
    Have you tried to compute the Kullback-Leindler divergence $$D_{KL}(Z | X) = \int p_Z(x)\frac{\log(p_Z(x))}{\log(p_X(x))},dx, $$ where $Z$ is the standard Gaussian and $X$ is the Gaussian with covariance matrix $P$? – Deane Feb 09 '25 at 16:13

2 Answers2

18

I am not aware of the Kullback-Leibler divergence. Nonetheless, writing $x_1, \dots, x_n$ for the eigenvalues of $P$, using the AM-GM inequality

$$ \frac{x_1 + \dots + x_n}{n} \geq \left( x_1 \dots x_n \right)^{1/n} $$

gives

$$ \frac{\operatorname{tr} (P)}{n} \geq \det(P)^{1/n} $$

using the expression of the trace and determinant in terms of eigenvalues, hence

$$ \log \left( \frac{\operatorname{tr}(P)}{n} \right) \geq \frac{1}{n} \log( \det(P) ) $$

Next, use

$$ x - 1 \geq \log( x ), \quad \, \forall x \geq 0,$$

$$ \frac{1}{n} \operatorname{tr}(P) - 1 \geq \frac{1}{n} \log( \det(P) ) \quad \implies \operatorname{tr}(P) - n \geq \log( \det(P) ) $$

Thomas Lehéricy
  • 3,487
user7440
  • 1,112
3

Following Deane Yang's advice, the Kullback-Leibler divergence between normally distributed random variables ${\bf X} \sim \mathcal{N} \left( {\bf 0}_n, {\bf P} \right)$ and ${\bf Y} \sim \mathcal{N} \left( {\bf 0}_n, {\bf I}_n \right)$ is

$$ D_{\text{KL}} ({\bf X} \| {\bf Y}) = \dots = \frac12 \left( \operatorname{tr} ({\bf P}) - \log \det ({\bf P}) - n \right) $$

and, since $D_{\text{KL}} ({\bf X} \| {\bf Y}) \geq 0$,

$$\color{blue}{\operatorname{tr} ({\bf P}) - \log \det ({\bf P}) \geq n}$$

References

Rodrigo de Azevedo
  • 23,223