Gradient of $H \mapsto \| A − W H \|_{\text{F}}^2$

Question

What is the gradient of $f (H) = \| A − W H \|_{\text{F}}^2$?

Answer 1

Let $\phi(H) = \operatorname{tr} ((A-WH)^T (A-WH))$, then by taking the linear term in $\Delta$ in the expression $\phi(H+\Delta)-\phi(H)$ we see that $D \phi(H)(\Delta) = -2 \operatorname{tr} ((A-WH)^T W\Delta)$ and so the 'matrix' format is $-2W^T(A-WH)$.