1

I am proving the second order derivative in the Matrix Cookbook. This is my current work:

\begin{align} \frac{\partial}{\partial x_{pq}} tr(X^TBX)&=\frac{\partial}{\partial x_{pq}} \sum_{i=1}^n (X^TBX)_{ii}\\ &=\frac{\partial}{\partial x_{pq}} \sum_{i=1}^n \sum_{j=1}^m (X^TB)_{ij}X_{ji}\\ &=\frac{\partial}{\partial x_{pq}} \sum_{i=1}^n \sum_{j=1}^m \sum_{k=1}^m X^T_{ik} B_{kj} X_{ji}\\ &=\frac{\partial}{\partial x_{pq}} \sum_{i=1}^n \sum_{j=1}^m \sum_{k=1}^m B_{kj}(x_{ki} x_{ji})\\ % &=\frac{\partial}{\partial x_{pq}} B_{pp} x_{pq}^2+\frac{\partial}{\partial x_{pq}} \sum_{j=1}^m B_{pj} x_{jq}+\frac{\partial}{\partial x_{pq}} \sum_{k=1}^m B_{kp} x_{kq}\\ &=\underbrace{\frac{\partial}{\partial x_{pq}} B_{pp} x_{pq}^2}_\text{$k=j=p,i=q$} +\ \underbrace{\frac{\partial}{\partial x_{pq}} \sum_{j=1}^m B_{pj} x_{pq} x_{jq}}_\text{$p=k\neq j,i=q$}+\ \underbrace{\frac{\partial}{\partial x_{pq}} \sum_{k=1}^m B_{kp} x_{kq} x_{pq}}_\text{$p=j\neq k,i=q$}\\ &=2B_{pp} x_{pq}+ \sum_{j=1}^m B_{pj} x_{jq} +\sum_{k=1}^m B_{kp} x_{kq} \end{align}

I saw another post that solved it, but why did the poster just ignore the $\frac{\partial}{\partial x_{pq}} B_{pp} x_{pq}^2$ term?

Reungu Ju
  • 11
  • 1
    There is no such term: you have to differentiate wrt $x_{pq}$, hence you get $2B_{pp}x_{pq}$, and two sums on $j\ne p$ and $k\ne p$ respectively: just complete each sum with one $B_{pp}x_{pq}$ and you are done. – Jean-Claude Arbaut Apr 17 '23 at 22:32
  • Sorry, there was a mistake I forgot to rectify while copying my previous latex code, it should be $2B_{pp}x_{pq}$. But then I got $2(BX){pq}+BX{pq}+(B^TX){pq}$. How does the $2(BX){pq}$ go away? – Reungu Ju Apr 17 '23 at 22:40
  • I also addressed this in my comment. On the last line you have sums from $1$ to $m$, while on the preceding line you have sums from $1$ to $m$, with the extra condition $j\ne p$ and $k\ne p$ respectively, i.e. each sum has a missing term. This missing term is exactly $B_{pp}x_{pq}$, in each sum. Therefore, the $2B_{pp}x_{pq}$ term in the last line has actually disappeared in the two sums, and shouldn't be here. – Jean-Claude Arbaut Apr 17 '23 at 22:43
  • I understand. Thank you! – Reungu Ju Apr 17 '23 at 22:49
  • Set $B=I$ in this related post – greg Apr 18 '23 at 23:33

0 Answers0