2

Given $A^TD_iA$, where:

  • $A$ is a rectangular matrix
  • $D_i$ is a diagonal matrix with positive entries
  • $A^TD_iA$ is a positive definite matrix

I need to compute the eigendecompositions of $A^TD_iA$, where $i\in{1,...,N}$. Each $D_i$ is different. Is there a way to do all these eigendecompositions without doing it from scratch for each $i$?

Edit: In this problem, $D_i$ is given by

$$D_i:=\left(\left(\text{diag}(b-Ax_i)\right)^{2}\right)^{-1}$$

where $b$ is a fixed column vector, and $x_i$ is a column vector different for each $i$. The operator $\text{diag}(\cdot)$ takes a column vector and returns a diagonal matrix whose diagonal elements are the elements of that vector.

Lab
  • 65
  • First of all, it bears mentioning that in the case where $AA^T$ is diagonal (i.e. $A$ has orthogonal rows), this is very easy – Ben Grossmann Feb 24 '23 at 20:55
  • Second, note that this is equivalent to computing the singular value decomposition of $\sqrt{D_i}A$. If $D_2$ is the same as $D_1$ except for one diagonal entry, then we can obtain the singular value decomposition of $\sqrt{D_2}A$ via a rank-1 update to the SVD of $\sqrt{D_1}A$. Some algorithms for doing this are discussed on this MO post. – Ben Grossmann Feb 24 '23 at 21:01
  • It might also be helpful to note that the square roots of the eigenvalues of $A^TDA$ match the positive eigenvalues of $$ \pmatrix{0&A^T\sqrt{D}\ \sqrt{D}A & 0} \text{ or } \pmatrix{0&\sqrt{D}A\ A^T\sqrt{D} & 0} $$ – Ben Grossmann Feb 24 '23 at 21:05
  • If we want to minimize the number of entries of the matrix that depend on $D$, you could also use $$ \pmatrix{0&A^T\ DA & 0} \text{ or } \pmatrix{0&DA\ A^T& 0} $$ instead – Ben Grossmann Feb 24 '23 at 21:13
  • Thanks, I have just added the definition of each $D_i$, in case the structure of this matrix could be exploited – Lab Feb 27 '23 at 07:59

1 Answers1

2

Let $A$ be $m \times n$, so $D$ is $m \times m$ and $A^T D A$ is $n \times n$. If $m \geq n$, I don't see any useful shortcuts.

If $m<n$, compute a $QR$-factorization $A = RQ$, so $Q$ is $n \times n$ orthogonal and $R$ is of the form $[R_0 \ 0]$ with $R_0$ an $m \times m$ lower triangular matrix. Then $$A^T D A = Q^T R^T D R Q = Q^{-1} \begin{bmatrix} R_0^T D R_0 &0 \\ 0&0 \end{bmatrix} Q.$$ Conjugating by $Q$ doesn't change the eigenvalues, so the eigenvalues are the same as those of the middle matrix, meaning $n-m$ zeroes and the eigenvalues of $R_0^T D R_0$.

Computing the $QR$ factorization is a one time cost, where as the improvement from $A^T D_i A$ to $R_0^T D_i R_0$ happens for each $i$ so, if you have a lot of diagonal matrices, and $m \ll n$, I would expect this to be a good trade off.

David E Speyer
  • 64,196