A question on positive semi definite matrices

Question

Suppose $A,B$ are symmetric, positive semi-definite matrices of same order such that $A \preceq B \preceq \kappa A$. How to prove that this is equivalent to $\frac{1}{\kappa}A^+ \preceq B^+ \preceq A^+$? Here $A^+$ is the generalized inverse of $A$.

Answer 1

If $A^+$ is in particular the Moore Penrose inverse, then we can proceed as follows: There exists an orthogonal matrix $U$ such that $$ \tilde A = U^TAU = \pmatrix{A_0 & 0\\0 & 0} $$ where $A_0$ is invertible (i.e. strictly positive definite). We now have $$ A \preceq B \preceq \kappa A \implies\\ \tilde A \preceq \tilde B \preceq \kappa \tilde A \implies\\ U^TAU \preceq U^TBU \preceq \kappa U^TAU \implies\\ \pmatrix{A_0 & 0\\0 & 0} \preceq \pmatrix{B_{11} & B_{12}\\B_{21} & B_{22}} \preceq \kappa \pmatrix{A_0 & 0\\0 & 0} $$ Now, from the right-side inequality, we have $$ \pmatrix{A_0 & 0\\0 & 0} - \pmatrix{B_{11} & B_{12}\\B_{21} & B_{22}} = \pmatrix{A_0 - \kappa B_{11} & -B_{12}\\ -B_{21} & -B_{22}} \succeq 0 \tag{RHS} $$ and from the left-side inequality, we have $$ \pmatrix{B_{11} & B_{12}\\B_{21} & B_{22}} - \pmatrix{A_0 & 0\\0 & 0} = \pmatrix{B_{11} - A_0 & B_{12}\\ B_{21} & B_{22}} \succeq 0 \tag{LHS} $$ Now, since any principal submatrix of a positive semidefinite matrix is also positive semidefinite, we have $B_{22} \succeq 0$ and $-B_{22} \succeq 0$, which must mean that $B_{22} = 0$. Since $\tilde B$ is positive semidefinite, we have $$ \pmatrix{x^T & y^T} \pmatrix{B_{11} & B_{12}\\B_{21} & 0}\pmatrix{x\\y} \geq 0 \implies\\ x^TBx - 2x^TB_{12}y \geq 0 $$ Noting that the above must hold for all choices of $x,y \in \Bbb R^n$, conclude that $x^TB_{12} = 0$ for all $x \in \Bbb R^n$, which is to say that $B_{12} = B_{21}^T = 0$. Thus, we have $$ \tilde B = U^TBU = \pmatrix{B_{11} & 0\\0 & 0} $$ Now, it suffices to show that $\tilde A^+ \preceq \tilde B^+ \preceq \kappa \tilde A^+$. To that effect, note that $$ A_0 \preceq B_{11} \preceq \kappa A_0 $$ is an inequality of strictly positive definite matrices. Thus, we have $$ A_0^{-1} \succeq B_{11}^{-1} \succeq (\kappa A_0)^{-1} = \frac 1 \kappa A_0^{-1} $$ Finally, we have $$ \frac 1{\kappa}\tilde A^+ = \pmatrix{\frac 1 \kappa A_0^{-1} & 0\\0 & 0} \preceq \tilde B^+ = \pmatrix{B_{11}^{-1} & 0\\0 &0} \preceq \tilde A^+ = \pmatrix{A_0^{-1} & 0\\0 & 0} $$ Thus, we can reach the desired conclusion. In particular: $$ \frac 1{\kappa} A^+ = U[\frac 1{\kappa}\tilde A^+]U^T \preceq \tilde B^+ = UB^+U^T \preceq A^+ = U\tilde A^+ U^T $$

---

Regarding the comment:

Lemma: if $A \preceq B$ are positive definite matrices, then $A^{-1} \succeq B^{-1}$

Proof: Note that $$ A \preceq B \implies\\ B^{-1/2}AB^{-1/2} \preceq I \implies\\ \text{all eigenvalues of } B^{-1/2}AB^{-1/2} \text{ are less than } 1 \implies\\ \text{all eigenvalues of } [B^{-1/2}AB^{-1/2}]^{-1} \text{ are more than } 1 \implies\\ B^{1/2}A^{-1}B^{1/2} \succeq I \implies\\ A^{-1} \succeq B^{-1} $$

Answer 2

Using Schur complement:

$$A\leq B\Leftrightarrow \left[\begin{array}{cc}A&I\\I&B^{-1}\end{array}\right]\leq 0\Leftrightarrow B^{-1}-A^{-1}\leq0 \Leftrightarrow B^{-1}\leq A^{-1}$$

$$B\leq \kappa A\Leftrightarrow \left[\begin{array}{cc}B&I\\I&\frac{1}{\kappa}A^{-1}\end{array}\right]\leq 0\Leftrightarrow \frac{1}{\kappa} A^{-1}-B^{-1}\leq0 \Leftrightarrow \frac{1}{\kappa} A^{-1}\leq B^{-1}$$