0

In an exercise for uni I am asked to prove that $w^* \in \mathbb{R}^d$ is a solution to $\min_{w \in \mathbb{R}^d}\lVert y - Xw \rVert_{2}^{2}$ if and only if $X^\dagger X w^* = X^\dagger y$, where $X^\dagger$ is the pseudo-inverse of $X$ (https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse).

My initial idea of solving this was by writing out the L2-norm expression, but I do not think this would help me, as this does not lead to an expression containing $X^\dagger$.

Could any of you help me with starting off this proof? thanks in advance!!

Jord van Eldik
  • 379
  • 1
    use the SVD definition of the pseudoinverse – Exodd Nov 26 '23 at 14:31
  • 1
    Or use that $w$ is a solution iff $X^\dagger y - w\in Ker(X)$ – Exodd Nov 26 '23 at 14:39
  • 1
    In general there are infinitely many generalized inverse, each one will yield a solution to the OLS problem $w_g:=(A^A)^gA^y$. The Moore-Penrose pseudo inverse is a particular that gives a solution that has minimal norm $w_+=(A^A)^+A^y= A^+y=\operatorname{arg.min}{|x|_2: x=\operatorname{arg.min}|Ax-y|_2}$. Here is an explanation. – Mittens Nov 26 '23 at 18:01

0 Answers0