Proof of inequality in positive semidefinite matrix

Question

Let $X \in \Bbb R^{n \times p}$ and $A \in \Bbb R^{k \times p}$, where $k<p$ and $\mbox{rank} (A)=k$, be arbitrary matrices. Prove that $$\left( X^T X \right)^{-1} \geq \left( X^T X \right)^{-1}A^T \left( A \left( X^T X \right)^{-1} A^T \right)^{-1} A \left( X^T X \right)^{-1}.$$ Assume that $X^T X$ and $A \left(X^T X \right)^{-1}A^T$ are invertible.

---

My idea is to use orthogonal diagonalization on both sides since both sides are positive semidefinite matrices. However, I still have no idea what to do next.

Answer 1

Let $B=\sqrt{(X^TX)^{-1}}A^T$, then we only need to show $I\succeq B(B^TB)^{-1}B^T$. Use singular value decomposition, $B=UDV$ and {$s_i$} are the singular values, then $(B^TB)^{-1}=V^Tdiag\{s_1^{-2},\cdots,s_k^{-2},0,\cdots,0\}V$, so RHS=$Udiag\{1,\cdots,1,0,\cdots,0\}U^T\preceq I$.