Let the scalar field $\Phi : \mathbb{R}^{m \times r} \times \mathbb{R}^{r \times n} \to \mathbb{R}$ be defined by
$$\Phi(X,Y) := \sum_{i=1}^m \sum_{j=1}^n \exp \left( - \left( \frac{XY-A}{\gamma} \right)_{ij}^2 \right) $$
where $A\in \mathbb{R}^{m\times n}$ and $\gamma \in \mathbb{R}$ are given. I would like to compute the gradients $\nabla_X \Phi$ and $\nabla_Y \Phi$.
Could anyone please help me with the above differentiation please? I would appreciate a lot.
Define some auxiliary matrices and calculate their differentials $$\eqalign{ B &= \tfrac{1}{\gamma}(XY-A) &\implies dB=\tfrac{1}{\gamma}(X\,dY+dX\,Y) \cr C &= -B\odot B &\implies dC=-2B\odot dB \cr E &= \exp(C) &\implies dE=E\odot dC \cr F &= -\tfrac{2}{\gamma}B\odot E \cr }$$ where the symbol $(\odot)$ represents the elementwise/Hadamard product, and the exp() function is understood to be applied elementwise.
Write the objective function in terms of these new variables.
Then calculate its differential and gradients.
$$\eqalign{
\Phi &= J:E \cr
d\Phi
&= J:dE \cr
&= J:(E\odot dC) \cr
&= E:dC \cr
&= E:(-2B\odot dB) \cr
&= -2(B\odot E):dB \cr
&= \gamma F:\tfrac{1}{\gamma}(X\,dY+dX\,Y) \cr
&= X^TF:dY + FY^T:dX \cr
\frac{\partial\Phi}{\partial Y} &= X^TF, \quad
\frac{\partial\Phi}{\partial X} = FY^T \cr
}$$
where $J\in{\mathbb R}^{m\times n}$ is a matrix of all ones, and (:) represents the trace/Frobenius product, i.e.
$$\eqalign{ A:B = {\rm Tr}(A^TB)}$$
The cyclic property of the trace allows the terms to be rearranged in various ways.
$$\eqalign{
A:BC = AC^T:B = B^TA:C
}$$
Finally, the Hadamard and Frobenius products commute with themselves and each other
$$\eqalign{
A:B &= B:A \cr
B\odot C &= C\odot B \cr
A:(B\odot C) &= (A\odot B):C \cr
}$$
