How to find the gradient of $X \mapsto a^T X b$?

Question

$\nabla_X (a^TXb)$ where $a$ and $b$ are vectors, and $X$ is a positive matrix. How to solve this?

Answer 1

$f(X) = a^T X b; \tag 1$

$f(X + H) = a^T(X + H)b = a^T X b + a^T H b; \tag 2$

$f(X + H) - f(X) = a^T H b; \tag 3$

$f(X + H) - f(X) - a^T H b = 0; \tag 4$

since

$\displaystyle \lim_{\Vert H \Vert \to 0} \dfrac{0}{\Vert H \Vert} = 0, \tag 5$

it follows that the linear map

$H \to a^T H b = f(H) \tag 6$

is the derivative of the linear map (1) with respect to $X$, a result which squares with the well-known fact that the derivative of a linear map is the map itself.

Answer 2

$$f (\mathrm X) := \mathrm a^\top \mathrm X \,\mathrm b = \mbox{tr} \left( \mathrm a^\top \mathrm X \,\mathrm b \right) = \mbox{tr} \left( \mathrm b \mathrm a^\top \mathrm X \right) = \langle \mathrm a \mathrm b^\top, \mathrm X \rangle$$

Hence,

$$\nabla_{\mathrm X} \, f (\mathrm X) = \mathrm a \mathrm b^\top$$