0

Given the set of vectors $\{\mathbf{g}^{1}, \ldots, \mathbf{g}^{N-1} \}$ where $\mathbf{g}^{i} \in \mathbf{R}^M$. Assume that $N \leq M$ and elements of $\mathbf{g}^{i}$ follows normal distribution, i.e. ,$\mathbf{g}^{i}_{m} \sim \mathcal{N}(0,1)$.

I would like to compute the probability that a new vector $\mathbf{g}^{N} \in \textrm{span} \{\mathbf{g}^{1}, \ldots, \mathbf{g}^{N-1} \} $.

Do you have any solution or suggestion?

Hoping_Blessing
  • 417

1 Answers1

2

This probability is $0$. You can find it by conditioning on $g_1,\dots,g_{N-1}$. $span(g_1,\dots,g_{N-1})$ is an $N-1$ dimensional subspace. Let $S$ be the subspace orthogonal to this and pick a line $L$ in $S$. Observe that $L$ always exists since $N\leq M$.

Due to rotational invariance of normal distribution, the projection of $g_N$ onto $L$ is a standard normal variable hence projection is nonzero almost surely. Since $g_N$ has nonzero projection it cannot be in the set $span(g_1,\dots,g_{N-1})$. The reason is by construction all elements of this set are orthogonal to $L$.

Hence, for an realization of $g_1,\dots,g_{N-1}$, \begin{equation} P(g_N\in span(g_1,\dots,g_{N-1}))=0 \end{equation} Integrating the conditional probability over $g_1,\dots,g_{N-1}$ you can conclude that total probability is $0$ as well.

ecstasyofgold
  • 884
  • Thank you very much. I almost understand your idea. One thing I have not understood is that "Due to rotational invariance of normal distribution, the projection of $g_N$ onto $L$ is a standard normal variable hence projection is nonzero almost surely". Can you give me some reference or can you please explain in more detail? – Hoping_Blessing Feb 09 '16 at 10:28
  • Rotational invariance means that you can freely take the line $L$ to be whatever you want and behavior is the same. In particular, take $L$ to be the span of $e_1=[1~0~0~\dots~0]$. When you do this, the inner product is simply $e_1^*g_N$ which is the first coordinate of $g_N$ which has normal distribution. This is nonzero almost surely.

    You can read formal description and why normal distribution is rotationally invariant here: http://math.stackexchange.com/questions/1074218/what-does-rotational-invariance-mean-in-statistics

     – ecstasyofgold Feb 09 '16 at 10:34
  • Rotational invariance essentially follows from the fact that density function of a standard Gaussian vector depends only on the length and not the direction. – ecstasyofgold Feb 09 '16 at 10:40
  • Thank you very much! I will investigate more. 1 vote for you ! – Hoping_Blessing Feb 09 '16 at 10:42