Are the projections along orthogonal direction of multivariate normal distribution with diagonal covariance matrix independent?

Question

I'm taking a probability class and my prof used the following theorem IIRC.

Let $g\sim\mathcal{N}(\mu,\Sigma)$ where $\Sigma$ is diagonal( I don't know if this condition is necessary) and $\langle u,v\rangle=0$, then $\langle g,u\rangle$ and $\langle g,v\rangle$ are independent.

Is this correct? If so, how to prove this? I believe the following is a special case of the theorem: Are the random variables $X + Y$ and $X - Y$ independent if $X, Y$ are distributed normal? I tried to use the same technique to prove the theorem but got stuck.

If $\color{blue}{ u^T \Sigma v = 0}$, then you can show that $u^T g$ and $v^T g$ are independent. It won't generally hold if just $u^T v = 0$. Hint for showing this: write $z := (u^Tg, v^Tg)^T$ in the form $Ag$ for some matrix $A$, and use that the covariance matrix of $Ag$ is $A\Sigma A^T$. Also recall facts about linear transformations of normal random vectors and when such a vector has independent components based on its covariance matrix). — Minus One-Twelfth, Feb 17 '19 at 02:54

angryavian · Answer 1 · 2019-02-17T04:12:41.973

0

Elaborating on Minus One-Twelfth's comment:

The pair $(g^\top u, g^\top v)$ is jointly normal. (Why?)

Thus independence is equivalent to $\text{Cov}(g^\top u, g^\top v) = 0$. The covariance is $$\begin{align}\text{Cov}(g^\top u, g^\top v) &= E[(g^\top u - E[g^\top u])(g^\top v - E[g^\top v])] \\ &= E[((g - \mu)^\top u)((g - \mu)^\top v)] \\ &= E[u^\top (g - \mu)(g-\mu)^\top v] \\ &= u^\top E[(g-\mu)(g-\mu)^\top] v \\ &= u^\top \Sigma v. \end{align}$$

If $\Sigma$ is a multiple of the identity matrix and if $\mu = 0$, then independence is equivalent to $u^\top v = 0$. However, in general you have to use the above expression $u^\top \Sigma v$.

edited Feb 17 '19 at 04:12

answered Feb 17 '19 at 02:59

angryavian

93,534

Why is it jointly normal? Is this a property of the multivariate Gaussian variable? I know that $g^Tu$ and $g^Tv$ are both normal. – Yihan Zhou Feb 17 '19 at 03:36
@YihanZhou One way to check that it is jointly normal is to check that any linear combination $a(g^\top u) + b (g^\top v)$ is [univariate] normal. Indeed, any such linear combination is itself a linear combination of the components of $g$. – angryavian Feb 17 '19 at 03:40
Why $E[(u^Tg)(g^Tv)]=u^T\Sigma v$? Sorry I'm not stat major and don't know much about multivariate Gaussian. – Yihan Zhou Feb 17 '19 at 04:04
@YihanZhou See my edit; I made a mistake earlier. – angryavian Feb 17 '19 at 04:13

score 0 · Answer 2 · answered Mar 18 '25 at 07:14

The statement is false in general. Consider $a\neq b$ and $Z_1,Z_2$ independent and $N(0,1)$, $g=(aZ_1,bZ_2)$, $0<\theta<\pi/2$, $u=(\cos \theta,-\sin \theta)$ and $v=(\sin \theta,\cos\theta).$ Then $\langle u,v\rangle=0$, and $$E(\langle g,u\rangle \langle g,u\rangle )=(a-b)\cos\theta \sin \theta \neq 0$$

Are the projections along orthogonal direction of multivariate normal distribution with diagonal covariance matrix independent?

2 Answers2