3

I am examining properties of mass moments of probability densities:

$$ m_{i}\equiv\int_{-\infty}^{\infty}x^{i}f\left(x\right)dx $$

Define a $\,n\times n\,$ covariance of the first $\,n\,$ moments:

$$ M_{ij}\equiv\int_{-\infty}^{\infty}f\left(x\right)\left(x^{i}-m_{i}\right)\left(x^{j}-m_{j}\right)dx=m_{i+j}-m_{i}m_{j} $$ where $1\leq i\leq n$ and $1\leq j\leq n$

I suspect but have failed to prove that:

$$ \left|M_{n\times n}\right|=0\iff\left\{ \begin{array}{c} f\left(x\right)=\frac{1}{r}\sum_{j=1}^{r}\delta\left(x-x_{j}\right)\\ r\leq n\end{array}\right\} $$

In other words if $M$ is $n\times n$ and its Determinant is $0$, then $f(x)$ is a set of $n$ or fewer spikes.
If $f(x)$ has more than $n$ spikes or has any continuous portion then $|M|>0$ for all $n$.

It's easy enough to demonstrate for chosen values of $r$ that $|M|=0$ ; the Mathematica code below does that. But I need a proof that if $\,|M|=0$ , then $\,f(x)\,$ consists of $\,r\,$ spikes, where $\,r\le n$ .

(*Demonstrate that for any n, Det[M]==0 if r==n, Det[M]>0 if r>n *)

n = 4;
M = Table[m[i + j] - m[i]*m[j], {i, 1, n}, {j, 1, n}];
Print[n, "-particle Det = ", Det[M] /. m[q_] -> (Sum[x[i]^q, {i, 1, n}]/n) // Simplify];
Print[n + 1, "-particle Det = ", Det[M] /. m[q_] -> (Sum[x[i]^q, {i, 1, n + 1}]/(n + 1)) // Simplify];
Jerry Guern
  • 2,764

2 Answers2

1

It's false as stated: the determinant should be $0$ for any $f$ that is supported on a set of cardinality $\le n$, not just a uniform distribution. In fact, let $X$ be a random variable whose distribution is supported on a set $\{r_1, \ldots, r_n\}$ of cardinality $n$. Let $P(z) = \prod_{k=1}^n (z - r_k) = \sum_{j=0}^n a_j z^j$. If ${\bf a} = (a_1, \ldots, a_n)^T$, then $P(X) = 0$ so $(M {\bf a})_j = \text{Cov}(X^j, P(X)) = 0$ (the lack of an $a_0$ term does not matter because $\text{Cov}(X^j, 1) = 0$) and therefore $M$ is singular.

Conversely, if $M$ is singular, take ${\bf b} \ne 0$ in its null space, and define $P(z) = \sum_{k=1}^n b_j z^j$. We have $\text{Var}(P(X)) = \text{Cov}(P(X),P(X)) = {\bf b}^T M {\bf b} = 0$, which implies $P(X) = 0$ a.s., and that means $X$ is almost surely one of the roots of $P$ (of which there are at most $n$).

Robert Israel
  • 470,583
  • A very dense answer! Thank you. I gather that you're saying M will be singular even if the spikes in the pdf have unequal weight. But I'm struggling with this step in your explanation: $(M {\bf a})j = \text{Cov}(X^j, P(X)) = 0$. Could you give that a bit of explanation? Also, I'm not certain if your answer proves $M{n\times n}$ is non-singular if f(x) has more than n spikes. – Jerry Guern Sep 13 '15 at 16:07
  • Covariance is bilinear, so $\text{Cov}(X^j, P(X)) = \sum_{k=0}^n a_k \text{Cov}(X^j, X^k) = \sum_{k=1}^n M_{jk} a_k = (M {\bf a})_j$. – Robert Israel Sep 13 '15 at 16:17
  • Thank you! It took me some time to understand this, hence the delay in awarding the bounty. – Jerry Guern Sep 14 '15 at 04:21
  • I posted a more baby-steps version of this answer below. – Jerry Guern Sep 14 '15 at 04:30
  • One more small question. What does "almost surely" mean in math? I can't even find a good online explanation, and I've looked. – Jerry Guern Sep 17 '15 at 02:05
  • It means "with probability $1$". – Robert Israel Sep 17 '15 at 14:28
  • I posted another bounty question that might be up your alley. http://math.stackexchange.com/questions/1455526/how-to-solve-an-integral-with-a-gaussian-mixture-denominator – Jerry Guern Oct 01 '15 at 12:15
1

I posted the Question and awarded the bounty to @RobertIsrael for his great Answer above. But I had to think pretty hard about his answer and "translate" into simpler language for myself, so I thought I'd post my simpler version.

First of all, my assertion wasn't quite correct because the spikes in the pdf $f(x)$ can be differently weighted. So the corrected theorem is:

$$ \left|M_{n\times n}\right|=0\iff\left\{ \begin{array}{c} f\left(x\right)=\sum_{j=1}^{r}w_{j}\delta\left(x-x_{j}\right)\\ r\leq n\end{array}\right\} $$

Since any weight can be zero, the case where there are fewer than n spikes is covered. The RHS of the above eqn can be restated as:

$$ f\left(x\notin\left\{ x_{1},...,x_{n}\right\} \right)=0 $$

Now consider this n-th order polynomial:

$$ q\left(x\right)\equiv\prod_{j=1}^{n}\left(x-x_{j}\right)=\sum_{k=0}^{n}a_{k}x^{k}$$

It is trivial but not necessary to calculate the $a_k$'s. For any x, either $f(x)$ or $q(x)$ will equal 0. Therefore, the following integrand and integral both equal zero:

$$ 0=\int f\left(x\right)\left(x^{j}-m_{j}\right)q\left(x\right)dx=\int f\left(x\right)\left(x^{j}-m_{j}\right)\sum_{k=0}^{n}a_{n}x^{k}dx=\sum_{k=0}^{n}a_{n}\left(m_{j+k}-m_{j}m_{k}\right)=\left(M_{n\times n}a\right)_{j} $$

This proves that $a$ is a 0-eigenvector of M, which can only occur if M is singular, QED: $$ f\left(x\notin\left\{ x_{1},...,x_{n}\right\} \right)=0\Rightarrow|M_{n\times n}|=0 $$

Proof of the converse: If $M$ is singular, then there exists some $a\neq0$ such that $Ma=0$, from which we can construct a polynomial $q\left(x\right)\equiv\sum_{k=0}^{n}a_{k}x^{k}$, the covariance of which is:

$$Cov\left(q\left(x\right)\right)=\int f\left(x\right)\left(q\left(x\right)-\left\langle q\left(x\right)\right\rangle \right)^{2}=\int f\left(x\right)\left(\sum_{k=0}^{n}a_{k}\left(x^{k}-m_{k}\right)\right)^{2}=a^{T}Ma=0$$

$Cov(q(x))=0$ implies that $f(x)=0$ whenever $q(x)\neq0$, i.e., $f(x)$ can only be non-zero at the n roots of $q(x)$, QED:

$$|M_{n\times n}|=0\Rightarrow f\left(x\notin\left\{ x_{1},...,x_{n}\right\} \right)=0$$

...which completes the proof.

Jerry Guern
  • 2,764