0

I have a very basic question. Suppose that $M$ is an $n \times n$ matrix with entries given by

$$M_{ij} = \rho^{|i-j|}$$

for some $\rho \in (0,1)$. Is it true that the matrix $M$ is non-negative definite (or even positive definite)?

I was trying to write $M$ as $A^\top A$ for some matrix $A$, but could not do so. Any help would be greatly appreciated.

Rodrigo de Azevedo
  • 23,223
Probabilist
  • 418
  • For small enough $\rho$ (depending on $n$, but $\rho<1/3$ is sufficient regardless of $n$), you can use Gerschgorin's theorem. Asymptotically as $n \to \infty$ you can use Szego's theorem which works out regardless of $\rho$. I don't know how to handle $\rho \geq 1/3$ for finite $n$, but numerically it seems to work. – Ian Jul 30 '21 at 15:23
  • $M$ can be regarded as a covariance matrix of an Ornstein–Uhlenbeck process, so it should be positive definite. I am looking to see if this can be proved in an elementary way. – Sangchul Lee Jul 30 '21 at 15:40
  • Related: https://math.stackexchange.com/q/2815525/339790 – Rodrigo de Azevedo Jul 30 '21 at 17:50

2 Answers2

4

It is positive definite. I will prove it for the case $n=3$ and the general case is similar. In this case \begin{aligned} &M=\pmatrix{1&\rho&\rho^2\\ \rho&1&\rho\\ \rho^2&\rho&1},\\ &\pmatrix{1&-\rho&0\\ 0&1&-\rho\\ 0&0&1}M\pmatrix{1&0&0\\ -\rho&1&0\\ 0&-\rho&1} =\pmatrix{1-\rho^2&0&0\\ 0&1-\rho^2&0\\ 0&0&1}.\\ \end{aligned} Therefore $M$ is positive definite because it is congruent to $(1-\rho^2)I_{n-1}\oplus1$.

In the general case, we have $(I-\rho J)M(I-\rho J)^T=(1-\rho^2)I_{n-1}\oplus1$ where $J$ denotes the upper triangular nilpotent Jordan block of size $n$. Thus $M=AA^T$ where \begin{aligned} A &=(I-\rho J)^{-1}\left(\sqrt{1-\rho^2}I_{n-1}\oplus1\right)\\ &=(I+\rho J+\rho^2 J^2+\cdots+\rho^{n-1}J^{n-1})\left(\sqrt{1-\rho^2}I_{n-1}\oplus1\right). \end{aligned}

user1551
  • 149,263
2

Motivated from my previous comment, we have a Cholesky decomposition $M = A^{\mathsf{T}}A$, where

$$ A = \begin{pmatrix} 1 & \rho & \rho^2 & \cdots & \rho^{n-1} \\ 0 & \gamma & \gamma \rho & \cdots & \gamma \rho^{n-2} \\ 0 & 0 & \gamma & \dots & \gamma \rho^{n-3} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & \gamma \end{pmatrix} $$

and $\gamma = \sqrt{1-\rho^2}$. For a proof, note that

$$ A_{ij} = \rho^{j-i} (\mathbf{1}_{\{i=1\}} + \gamma \mathbf{1}_{\{i > 1\}} ) \mathbf{1}_{\{i \leq j \}}. $$

Then

\begin{align*} [A^{\mathsf{T}}A]_{ij} = \sum_{k} A_{ki}A_{kj} &= \sum_{k} \rho^{i+j-2k} (\mathbf{1}_{\{k=1\}} + \gamma \mathbf{1}_{\{k > 1\}} )^2 \mathbf{1}_{\{k \leq \min\{i,j\} \}} \\ &= \sum_{k} \rho^{i+j-2k} (\rho^2 \mathbf{1}_{\{k=1\}} + 1-\rho^2 ) \mathbf{1}_{\{k \leq \min\{i,j\} \}} \\ &= \rho^{i+j} + (1-\rho^2) \sum_{k=1}^{\min\{i,j\}} \rho^{i+j-2k} \\ &= \rho^{i+j - 2\min\{i,j\}} \\ &= \rho^{|i - j|}, \end{align*}

which is precisely $M_{ij}$.

Sangchul Lee
  • 181,930
  • thanks for your answer! One question, I wanted to construct a Gaussian process (infinite sequence) with this covariance structure. Now that I know that the finite dimensional matrices are positive definite, I can apply Kolmogorov's consistency theorem to construct such a Gaussian process. But since you mentioned that this is the Ornstein-Uhlenbeck process, I am wondering if that is true. In the O.U. process, the covariance is something like $e^{-a|t-s|} - e^{-a(t+s)}$, right? How can we ignore the last $e^{-a(t+s)}$ term? – Probabilist Jul 30 '21 at 17:52
  • @Probabilist The particular version of O.U. process I have in my mind is $(X_t){t\in\mathbb{R}}$ $$X_t=\rho^t W(\rho^{-2t}), $$ where $(W{t\geq0})$ is the standard Brownian motion. Then this gives $$\operatorname{Cov}(X_s,X_t)=\rho^{|s-t|}.$$ The difference between the formula you mentioned and the formula we have here is whether you started the process from $t=0$ or from $t=-\infty$ (see here.) – Sangchul Lee Jul 30 '21 at 17:57