3

This is related to an attempt at solving this problem : Best rank-$1$ approximation of matrix with condition.

Let $M\in\mathbb R^{m\times m}$ be PSD symmetric and $a\in \mathbb R^m$ be such that $0\leq a$, i.e. for $i=1,\dots, m$, $a_i\geq 0$. I am trying to solve the following problem : \begin{align*} &\min_{\substack{b\in\mathbb R^m:\\0\leq b}} ~(a+b)^TM(a+b) \end{align*}

Since the problem is convex, the KKT conditions are both necessary and sufficient and are ($\mu$ are the Lagrange multipliers) \begin{align*} \begin{cases} 2M(a+b) = \mu\\ 0\leq b\\ 0\leq \mu\\ \mu^T b=0 \end{cases} \end{align*}

Is there any closed form for this problem ? or a fast and simple algorithm that optimize it ?

P. Quinton
  • 6,249
  • By solving the problem, you mean minimize the function $(a+b)^T M (a+b)$? At first sight I thought it was about solving the first equation. – Carlos Adir Mar 14 '23 at 16:18
  • @CarlosAdir Yes, the first equality is trivial, the optimization is not. – P. Quinton Mar 15 '23 at 13:46
  • @P.Quinton So, you know it is a convex problem which can be solved by some solvers such as CVX? – River Li Mar 16 '23 at 12:29
  • @RiverLi yes I know some solvers that can do that, including scipy.optimize. Those are however too slow on this problem, and I am trying to get a grasp at what kind of steps they propose. The original problem was some biconvex optimization and therefore just a good step for this problem is already good progress. – P. Quinton Mar 16 '23 at 12:45
  • Just curious: how large is $m$? – fedja Mar 16 '23 at 22:41
  • $M = Q\Lambda Q^T$; $b_i = \max {-[Q(Q^Ta)]_i, 0}$? – rumathe Mar 17 '23 at 11:09
  • @fedja $m$ is rather small, around $32$, the problem is that I launch this program a crasy amount of time on pytorch and therefore I would like to make it as simple as possible to not have to send data back and forth from GPU to CPU to another program etc... – P. Quinton Mar 17 '23 at 16:05
  • @rumathe Yes I tried something like that, but I think it is hard to show that $\mu = 2 M(a+b)$ is positive right ? If it is the case, then it is very nice. Also I think that $QQ^T=I$ right ? so it cannot be exactly that. – P. Quinton Mar 17 '23 at 16:10
  • @P.Quinton I tried to share a few thoughts about your main question in the linked thread. Let me know what you think of them. With $m$ as small as 32 I can even try to run some examples to see whether my proposed alternative approach is reasonably efficient. – fedja Mar 17 '23 at 18:55

0 Answers0