16

I have the Cholesky decomposition of a matrix $M$. However, I need the Cholesky decomposition of the inverse of the matrix, $M^{-1}$. Is there a fast way to do this, without first computing $M^{-1}$? In other words, is there a relationship between the Cholesky decompositions of a matrix and of its inverse?

My matrix is a covariance matrix and, hence, positive-definite.

Rodrigo de Azevedo
  • 23,223
user1971988
  • 261

3 Answers3

7

If $L^T L = R$ is the available Cholesky decomposition, then inverting both sides of the equation you get,

$$L^{-1}(L^{T})^{-1} = R^{-1} $$

And since transposition and inverse are interchangeable:

$$L^{-1}(L^{-1})^{T} = R^{-1} $$

So if you define $P = (L^{-1})^T$ this is your desired answer. In other words,

$$P^{T}P=R^{-1}$$

317070
  • 115
  • 6
nimrodm
  • 236
  • 14
    The decomposition obtained here for $R^{-1}$ is not the Cholesky decomposition as the first factor ,$P^T$, is upper triangular (should be lower triangular). – Juho Kokkala Jan 29 '17 at 12:58
  • @JuhoKokkala Why does that matter? It is still triangular. – Rodrigo de Azevedo Jul 11 '18 at 17:33
  • 3
    Whether it matters in practice would depend on what you use the Cholesky factor for (not specified in the question). The question asks how to get the Cholesky decomposition of $M^{-1}$, and this answer gives some decomposition that (in the general case) is not the Cholesky decomposition. – Juho Kokkala Jul 11 '18 at 18:30
  • 2
    In general, is it numerically more stable to take inverse, then choleksy decomp; or cholesky decomp, then inverse? – seeker_after_truth Feb 04 '23 at 22:13
3

I do not believe there is any way to do it without some $O(n^3)$ operation. Different choices can have different constants. In particular, assuming $M=LL^T$ and you already have the Cholesky factor $L$, computing $L^{-1}$ costs ~$n^3/3$ flops, and given that, you can form $M^{-1}$ = $L^{-T}L^{-1}$ in ~$n^3/3$ flops and take its Cholesky in another ~$n^3/3$ flops, which is ~$n^3/3$ flops cheaper than pretending you don't have $L$, forming $M^{-1}$ directly and taking its Cholesky. But it's still $O(n^3)$.

Rif A. Saurous
  • 41
  • Isn't inverting a triangular matrix O(n^2) instead of the O(n^3) of a matrix in general? Which would make the operation O(n^2) too, and thus sometimes worthwhile. – ninpnin Jun 19 '24 at 11:05
1

To add to previous answers, if we view $X$ as covariance matrix of data, the relationship between two decompositions reduces to relationship between coefficients of "right-to-left" autoregressive model and "left-to-right" one (details)

In general, switching order of "inverse" and "Cholesky" gives different results. IE

$$ \text{X=}\left( \begin{array}{ccc} 10 & 0 & 4 \\ 0 & 10 & -2 \\ 4 & -2 & 15 \\ \end{array} \right) $$

Using Cholesky and inverting factors, gives this decomposition

$$X^{-1}= \left( \begin{array}{ccc} 1 & 0 & -\frac{2}{5} \\ 0 & 1 & \frac{1}{5} \\ 0 & 0 & 1 \\ \end{array} \right)\left( \begin{array}{ccc} \frac{1}{10} & 0 & 0 \\ 0 & \frac{1}{10} & 0 \\ 0 & 0 & \frac{1}{13} \\ \end{array} \right)\left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -\frac{2}{5} & \frac{1}{5} & 1 \\ \end{array} \right) $$

While Cholesky on the inverse directly gives

$$X^{-1}= \left( \begin{array}{ccc} 1 & 0 & 0 \\ -\frac{4}{73} & 1 & 0 \\ -\frac{20}{73} & \frac{2}{15} & 1 \\ \end{array} \right),\left( \begin{array}{ccc} \frac{73}{650} & 0 & 0 \\ 0 & \frac{15}{146} & 0 \\ 0 & 0 & \frac{1}{15} \\ \end{array} \right),\left( \begin{array}{ccc} 1 & -\frac{4}{73} & -\frac{20}{73} \\ 0 & 1 & \frac{2}{15} \\ 0 & 0 & 1 \\ \end{array} \right) $$

notebook

Yaroslav Bulatov
  • 5,579