2

Given the function

$$f(N) = x_1^T M x_2 $$

where

  • $x_1 = Nv_1 $
  • $x_2 = Nv_2 $
  • $x_1, x_2, v_1, v_2$ are vectors with dimension $n \times 1$
  • $M$ and $N$ are matrices with dimension $n \times n$

what's the derivative of $f(N)$ with respect to $N$?

Rodrigo de Azevedo
  • 23,223
Lei Yu
  • 123
  • It maybe not the fastest and most elegant method, but it always works for these kind of things: Write explicitly the function $f$ in terms of the coefficients $N_{i,j}$ and compute each partial derivatives! – Surb Aug 13 '14 at 13:05
  • Here's the function: $$ f(N) =\sum_{i=1}^n\sum_{j = 1}^n\sum_{k=0}^n \sum_{l=0}^n M_{i,j}N_{i,k}N_{j,l}v_{1,k} v_{2,l}$$ – Surb Aug 13 '14 at 15:00

2 Answers2

2

Generalizing the problem slightly, to use matrix (instead of vector) variables, $X_k = N\cdot V_k$.

We can write the function as $$ \eqalign { f &= X_1 : M\cdot X_2 \cr } $$ Now take the differential and expand $$ \eqalign { df &= dX_1 : M\cdot X_2 + X_1 : M\cdot dX_2 \cr &= d(N\cdot V_1) : M\cdot X_2 + X_1 : M\cdot d(N\cdot V_2) \cr &= dN\cdot V_1 : M\cdot X_2 + X_1 : M\cdot dN\cdot V_2 \cr &= dN : M\cdot X_2\cdot V_1^T + M^T\cdot X_1\cdot V_2^T : dN \cr &= [M\cdot X_2\cdot V_1^T + M^T\cdot X_1\cdot V_2^T] : dN \cr &= [M\cdot N\cdot V_2\cdot V_1^T + M^T\cdot N\cdot V_1\cdot V_2^T] : dN \cr } $$ The derivative is therefore $$ \eqalign { \frac {\partial f} {\partial N} &= M\cdot N\cdot V_2\cdot V_1^T + M^T\cdot N\cdot V_1\cdot V_2^T \cr } $$ If you dislike the Frobenius product, you can change the above derivation to use the trace instead $$ \eqalign { A : B &= \text{tr}(A^T\cdot B) \cr } $$

lynne
  • 420
0

Rephrasing, let

$$ f ({\bf X}) := {\bf b}_1^\top {\bf X}^\top {\bf A} \, {\bf X} \, {\bf b}_2 = \operatorname{tr} \left( {\bf X}^\top {\bf A} \, {\bf X} \, {\bf b}_2 {\bf b}_1^\top \right) $$

Taking the gradient,

$$ \nabla_{{\bf X}} f ({\bf X}) = \cdots = \color{blue}{{\bf A} \, {\bf X} \, {\bf b}_2 {\bf b}_1^\top + {\bf A}^\top {\bf X} \, {\bf b}_1 {\bf b}_2^\top} $$

Related

Rodrigo de Azevedo
  • 23,223