0

This is a followup question from the previous post, where the following multivariate distribution function is given by:

$$\exp(-\frac12 \vec x^T\Sigma^{-1}\vec x)$$

Now, it was mentioned in the comments that if the $\Sigma$ was singular, such as follows:

$$\begin{bmatrix} 1 & 1\\ 1 & 1 \end{bmatrix}$$

then the

"distribution function however will be singular, having infinite weight along the diagonal and zero weight everywhere else"

I want to mathematically see how this is the case? How the diagonal is infinite weight and how the off diagonal are all 0's

GENIVI-LEARNER
  • 543
  • 1
    The variances and covariances are all the same in your example (implying the correlation is $1$). This only happens if $x_2=x_1+k$ with probability $1$ for some constant $k$, so a diagonal line on the plane, and the more general case of a singular $\Sigma$ corresponds probability $1$ being on a lower dimensional hyperplane in a hyperspace. I would not translate this to "infinite weight" myself, but instead say "all the weight". – Henry Feb 11 '20 at 15:58
  • Could you please elaborate what is "all the weight", I am trying to understand what is the notion of weight here. Also the equation you wrote, shouldnt it be simply $x_2=x_1$ implying perfect correlation? – GENIVI-LEARNER Feb 11 '20 at 16:06
  • 1
    Variances and covariances tell you about dispersion and correlation but not about location. In this 2-D example it says perfect correlation but that says nothing about the means being the same – Henry Feb 11 '20 at 16:09
  • Omg! I think you also answered my other question here which has been bothering me! So I thought that perfect correlation between random variables imply knowing value about one we get to know the value about the other, but in the other question I wrote this is not true because my initial assumption is wrong....right? – GENIVI-LEARNER Feb 11 '20 at 16:14
  • Your previous question says $0$ mean – Henry Feb 11 '20 at 16:16
  • yes i think mean was the source of confusion, If all random variables have 0 mean then full correlation imply that knowing value of one, the other is known as well as you can see in the first plot here but when the mean's of these random variables are not zero or not the same then knowing about the value of one variable does not reveal the value of other variable which it is perfectly correlated to....right? here [continued] – GENIVI-LEARNER Feb 11 '20 at 16:21
  • correlation only gives us the measure of increasing and decreasing with unit proportionality meaning they are directly promotional with proportionality constant as 1 ...right? – GENIVI-LEARNER Feb 11 '20 at 16:22
  • Correlation does not give scale relationships while variance and covariance do. If $X$ and $Y$ have correlation $\rho$ then so too do $2X$ and $7X$ – Henry Feb 11 '20 at 16:24
  • I didnt quite get your last comment. Correlation is basically covariance normalized so I was speaking of them interchangeably. What is $2X$ and $7X$? – GENIVI-LEARNER Feb 11 '20 at 16:28
  • 1
    I meant to say $2X$ and $7 Y$. My point is in your "normalized". So scale and location changes do not affect correlation. A correlation of $1$ does not allow you to say "a proportionality constant of $1$" unless you know the means and variances are identical. Meanwhile scale does affect covariance though location does not. – Henry Feb 11 '20 at 17:59
  • ok, looks like you have a good visual notion of what that matrix in the question is doing. I would really appreciate if you could briefly elaborate how were you able to perform converstion from the matrix to this 2=1+ with probability 1 for some constant ", also what is the notion of the weights here? – GENIVI-LEARNER Feb 11 '20 at 20:37
  • if off diagonal terms were say $0.5$ what would be your above statement. In other words _____ with probability __ for some constant? – GENIVI-LEARNER Feb 11 '20 at 20:39
  • If the covaraince matrix is not singular then $\Sigma^{-1}$ exists and you can have a continuous density, and in the case of a bivariate normal distribution you will have a continuous density, in which case that probability becomes $0$ and the issue you started with goes away – Henry Feb 11 '20 at 22:16
  • I am really eager to know why probability becomes 0 if covariance matrix is not singular. – GENIVI-LEARNER Feb 13 '20 at 09:19

0 Answers0