Given $A_i, B_i \in \mathbb{R}^{k \times d}$, minimize $\sum_{i} \lVert U A_i V^T - B_i \rVert_F^2 $ over orthogonal $U, V$.

Question

Given a collection of rectangular matrices $A_i, B_i \in \mathbb{R}^{k \times d}$ for $1 \leq i \leq n$, I am looking for an analytical solution for orthogonal matrices $U \in \mathbb{R}^{k \times k}$ and $V \in \mathbb{R}^{d \times d}$ with $U^T U = I_k$ and $V^T V = I_d$ that minimize \begin{align} \sum_{i = 1}^n \lVert U A_i V^T - B_i\rVert_F^2. \end{align}

In the case that $i=1$ the answer should reduce to something similar to this question after diagonalizing $A$ and $B$ via the SVD. I also understand that it may be the case that there are an infinity of solutions which is fine.

That said, I'm finding it tricky to attack this problem. One idea I've had is to linearize the system with the Kronecker product to equivalently(?) minimize \begin{align} \lVert W \, [ \textrm{vec}(A_1), \ldots, \textrm{vec}(A_n)] - [\textrm{vec}(B_1), \ldots, \textrm{vec}(B_n)] \rVert_F^2 \end{align} for orthogonal $W$ subject to the constraint that $W = V \otimes U$, but I don't know how to impose the latter.

Answer 1

$ \def\BR#1{\Big[#1\Big]} \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\trace#1{\op{Tr}\LR{#1}} \def\frob#1{\left\| #1 \right\|_F} \def\q{\quad} \def\qq{\qquad} \def\qif{\q\iff\q} \def\qiq{\q\implies\q} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\m#1{\left[\begin{array}{r}#1\end{array}\right]} \def\red#1{\color{red}{#1}} \def\CLR#1{\red{\LR{#1}}} \def\S{\sum_k} \def\A{A_k} \def\B{B_k} $The standard Procrustes problem can be solved using the SVD $$\eqalign{ \min_R\;\frob{AR-B}^2 \qiq USV^T=A^TB, \q R = UV^T \\ \min_Q\;\frob{QA-B}^2 \qiq USV^T=BA^T, \q Q = UV^T \\ }$$ This is easily extended to accommodate multiple matrices $$\eqalign{ \min_R\;\S\frob{\A R-\B}^2 \qiq USV^T=\S \A^T\B, \q R = UV^T \\ \min_Q\;\S\frob{Q \A-\B}^2 \qiq USV^T=\S \B\A^T, \q Q = UV^T \\ }$$ This suggests breaking your problem up into two related problems $$\eqalign{ \min_R\;\S\frob{\LR{Q\A}R-\B}^2 \qiq \S \A^TQ^T\B = USV^T, \q R = UV^T \\ \min_Q\;\S\frob{Q\LR{\A R}-\B}^2 \qiq \S \B R^T\A^T = USV^T, \q Q = UV^T \\ }$$ Initialize each rotation matrix to the respective identity matrix,
$\q$then hold $Q$ constant and update $R$,
$\q$then hold $R$ constant and update $Q$,
$\q$then hold $Q$ constant and update $R$,
$\q$etc.
Keep going until the matrices converge.