Find gradient of $\operatorname{tr} \left( B^{T}X^{T}A^{T}CXD \right)$

Question

I need to find the derivative wrt ${X}$ of:

$$ f({X}) = \operatorname{tr}( B^{T}X^{T}A^{T}CXD )$$

To make this simpler, I let $E = A^{T}C$, then I have:

$$ f({X}) = \operatorname{tr}( B^{T}X^{T}EXD ) = \operatorname{tr}( DB^{T}X^{T}EX )$$

We can rewrite this using the Frobenius inner product:

$$ f({X}) = XBD^{T}:EX$$

So we can calculate the differential easily now:

$$ df = (dX)BD^{T}:EdX$$

Unfortunately this is where I am stuck.

Perhaps we can rewrite this as:

$$ df = BD^{T}:(dX)^{T} E dX$$

But how do we isolate the dX value?

Edits:

I tried: $(dX)^{T} E dX = \operatorname{tr}( dXdX^{T}E) = \operatorname{tr}( EdXdX^{T}) $ but that isn't getting me anywhere either...

@J.G. Such that they work in the original formulation. However they are all matrices. — the_src_dude, Dec 13 '19 at 14:29
@Fimpellizieri No I have not. Perhaps I will try that as well, but I think this can be solved using differentials. I just don't know how...am I missing a step? — the_src_dude, Dec 13 '19 at 14:36
I'm not well versed in differentials. I usually deal with these problems by considering $(X+H)$, etc. — Fimpellizzeri, Dec 13 '19 at 14:37
@Fimpellizieri Neither am I haha I am just learning this now to try and solve some problems I am encountering. — the_src_dude, Dec 13 '19 at 14:47
You cannot differentiate both X's in one go, i.e. $d(xx)=(dx,x+x,dx)\ne(dx,dx)$ — greg, Dec 13 '19 at 15:14
@greg So then $tr(B^{T}X^{T}EXD) \rightarrow tr(B^{T}(dX^{T})EXD) + tr(B^{T}X^{T}E(dX)D) $? — the_src_dude, Dec 13 '19 at 16:07
Yes. It's the same thing that Joriki's saying in his answer. — greg, Dec 13 '19 at 17:10
There's no reason for $C$ to be there, as it could be absorbed by $A$. And $D$ could be absorbed by $B$. Take a look at this. — Rodrigo de Azevedo, Dec 14 '19 at 10:34
@RodrigodeAzevedo Almost, but regardless that was very helpful. Thank you for your kind assistance. I was hoping to do it using differentials. I am learning alot about this subject -- enough to start making some real progress on my engineering problems. I just hope my model is reasonable now haha but that is a different story! — the_src_dude, Dec 16 '19 at 11:10

score 1 · Accepted Answer · answered Dec 13 '19 at 14:51

1

You get two terms, one for each factor involving $X$. You can manipulate the trace for each one separately. Apparently you know that the derivative of $\operatorname{tr}Y^\top X$ is $Y$. For the first term, apply this to $\operatorname{tr}(DB^\top X^\top EX)$ to obtain $E^\top X BD^\top$. For the second term, use the invariance of the trace under cyclic permutation and under transposition to write $\operatorname{tr}(DB^\top X^\top EX)=\operatorname{tr}(X^\top EXDB^\top)=\operatorname{tr}(BD^\top X^\top E^\top X)$ and obtain the second term $EXDB^\top$. Thus the derivative is $E^\top X BD^\top+EXDB^\top$.

answered Dec 13 '19 at 14:51

joriki

242,601

Ah okay so I missed the two term part...I am not quite sure how to get that out yet though. Ie where the two terms come from. – the_src_dude Dec 13 '19 at 16:01
@the_src_dude: It's basically the product rule. If you take the derivative of $d\cdot b\cdot x\cdot e\cdot x$ with respect to $x$ for scalars, you likewise get $d\cdot b\cdot e\cdot x+d\cdot b\cdot x\cdot e$. The only difference is that you need to manipulate the trace in two different ways to see what exactly the resulting terms should be. – joriki Dec 13 '19 at 18:20

Find gradient of $\operatorname{tr} \left( B^{T}X^{T}A^{T}CXD \right)$

1 Answers1