Moore Condition for m x n matrix

Question

I'm not sure how to prove any of this. The only thing that seems to be related to this is: How does the SVD solve the least squares problem? but it does not answer any of the questions below

3.32. Prove that the pseudoinverse $\boldsymbol{A}^+$ of an $m\times n$ matrix $\boldsymbol A$, as defined using the SVD in Section 3.6.1, satisfies the following four properties, known as the Moore-Penrose conditions.

(a) $\boldsymbol{A}\boldsymbol{A}^+\boldsymbol{A}=\boldsymbol{A}$.

(b) $\boldsymbol{A}^+\boldsymbol{A}\boldsymbol{A}^+=\boldsymbol{A}^+$.

(c) $(\boldsymbol{A}\boldsymbol{A}^+)^T=\boldsymbol{A}\boldsymbol{A}^+$.

(d) $(\boldsymbol{A}^+\boldsymbol{A})^T=\boldsymbol{A}^+\boldsymbol{A}$.

Answer 1

$$ \mathbf{A} = \mathbf{U} \, \Sigma \, \mathbf{V}^{*}, \qquad \mathbf{A}^{+} = \mathbf{U} \, \Sigma^{+} \, \mathbf{V}^{*} $$

$\Sigma$ gymnastics

The problem distills down to an exercise in $\Sigma$ gymnastics. The diagonal matrix $\mathbf{S}$ contains the ordered singular values has size $\rho \times \rho.$ Note $\mathbf{S}^{T} = \mathbf{S}.$ The inverse matrix has reciprocal values on the diagonal.

First step: $$ \begin{align} \Sigma = \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{m\times n}, \qquad % \Sigma^{T} = \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{n\times m}, \qquad % \Sigma^{+} = \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{n\times m} % \end{align} $$ Second step: $$ \begin{align} \Sigma \, \Sigma^{T} = \left[ \begin{array}{cc} \mathbf{S}^{2} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{m\times m}, \qquad % \Sigma^{T} \, \Sigma &= \left[ \begin{array}{cc} \mathbf{S}^{2} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{n\times n} % \end{align} $$ Third step: $$ \begin{align} \Sigma \, \Sigma^{+} = \left[ \begin{array}{cc} \mathbf{I}_{\rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{m\times m}, \qquad % \Sigma^{+} \, \Sigma &= \left[ \begin{array}{cc} \mathbf{I}_{\rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{n\times n} % \end{align} $$ Fourth step: $$ \begin{align} \Sigma \, \Sigma^{+} \, \Sigma &= \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{m\times n} = \Sigma, \qquad % \Sigma^{+} \, \Sigma \, \Sigma^{+} = \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]_{n\times m} = \Sigma^{+} % \end{align} $$

---

a

$$ \begin{align} \mathbf{A} \, \mathbf{A}^{+} \, \mathbf{A} &= % A \left( \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \right) % A \left( \mathbf{V} \, \Sigma^{+} \, \mathbf{U}^{*} \right) % A+ \left( \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \right) \\ % & = \mathbf{U} \, \Sigma \, \Sigma^{+} \, \Sigma \, \mathbf{V}^{*} \\ % & = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\ % & = \mathbf{A} % \end{align} $$

b

$$ \begin{align} \mathbf{A}^{+} \mathbf{A} \mathbf{A}^{+} & = \mathbf{V} \, \Sigma^{+} \, \Sigma \, \Sigma^{+} \, \mathbf{U}^{*} \\ & = \mathbf{V} \, \Sigma^{+} \, \mathbf{U}^{*} \\ % & = \mathbf{A}^{+} % \end{align} $$

c

$$ \begin{align} \left( \mathbf{A} \, \mathbf{A}^{+} \right)^{*} &= % A \left( \mathbf{U} \, \Sigma \, \Sigma^{+} \, \mathbf{U}^{*} \right)^{*} \\ % A &= \left( \mathbf{U} \, \left[ \begin{array}{cc} \mathbf{I}_{\rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] \, \mathbf{U}^{*} \right)^{*} \\ &= \mathbf{U} \, \left[ \begin{array}{cc} \mathbf{I}_{\rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] \, \mathbf{U}^{*} \\ &= \mathbf{A} \, \mathbf{A}^{\dagger} % \end{align} $$

d

$$ \begin{align} \left( \mathbf{A}^{+} \, \mathbf{A} \right)^{*} &= % A \left( \mathbf{V} \, \Sigma^{+} \, \Sigma \, \mathbf{V}^{*} \right)^{*} \\ % A &= \mathbf{V} \, \left[ \begin{array}{cc} \mathbf{I}_{\rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] \, \mathbf{V}^{*} \\ &= \mathbf{A}^{+} \, \mathbf{A} % \end{align} $$

Answer 2

I'm going to assume that the SVD definition of the pseudo-inverse is $$ A^+ = V\Sigma^+U^*$$ where the SVD of $A$ is $A=U\Sigma V^*$ and $\Sigma^+$ is a diagonal matrix with diagonal entries $$[\Sigma^+]_{ii}=\left\{ \begin{array} 00 & \text{if} [\Sigma]_{ii}=0\\ ([\Sigma]_{ii})^{-1}&\text{otherwise}\end{array}\right.$$

It's then just a matter of plugging in the SVD of $A$ and $A^{+}$ into the equations (a)-(d). For example for (a) you have $$ AA^{+}A=(U\Sigma V^*)(V\Sigma^{+}U^*)(U\Sigma V^*)=U\Sigma\Sigma^+\Sigma V^*=U\Sigma V^*=A $$ $$