1

For my problem, I try to optimize the vector $\mathbf{w}\in\mathbb{C}^{N}$ at first. After exploiting semidefinite relaxation (SDR) method, the variable becomes $\mathbf{W} = \mathbf{ww}^H$ and the considered convex problem can be effectively solved with a PSD solution, as the rank constraint is dropped (the variable is set to be PSD). Then, how can we get a rank-1 solution from the solution of PSD matrix (probably rank > 1) and get back to a feasible solution of $\mathbf{w}$?

One way I heard from others is that to directly check the rank of the solution. If it happens to be rank 1, then we just choose the corresponding eigenvector to be the unique solution of $\mathbf{w}$.

But if it’s not rank-1, what do we usually perform to approximate a rank-1 solution from the solved PSD, that satifies the object and the constraints as much as possible? Meanwhile, I read a paper (reference 1 in this post) that also addresses this problem. In this paper, they cited an another paper [20] (reference 2 in this post) and said

[...] the rank of the output of the relaxed solution may be high (>1). We use standard SDP methods to encourage low rank solutions [20] and then approximate a feasible rank one solution from the output of problem $\mathcal{P}_8$ [21]

After I read the paper [20] they cited, I found the paper [20] is mainly solving the rank minimization problem, but I barely see how these methods can be exploited while the object and constraints of the original problem are met. And for [21] (reference 3 in this post), it's just the linear algebra book from G.Strang, so I guess it's something like eigenvalue decomposition for this step maybe.

References

  1. E. Aryafar and A. Keshavarz-Haddad, PAFD: Phased Array Full-Duplex [PDF], IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, Honolulu, HI, USA, 2018, pp. 261-269.

  2. M. Fazel, H. Hindi and S. Boyd, Rank minimization and applications in system theory [PDF], Proceedings of the 2004 American Control Conference, Boston, MA, USA, 2004, pp. 3273-3278 vol.4.

  3. G. Strang, Introduction to Linear Algebra [PDF]. Wellesley, MA: Wellesley-Cambridge Press, 2009.

tyrela
  • 363

1 Answers1

1

I know of two ways this is handled in practice, and unfortunately neither of them are super satisfying from a mathematical perspective.

  1. The first option is to take the solution of the SDP-relaxation, then project its solution onto the (nonconvex) set of rank-one matrices. This is performed (via Eckart-Young theorem) by just taking rank-one matrix which corresponds to the dominant singular value-vector pair of the original solution, and dropping the rest. It can be shown that this rank-one matrix is the "closest" (under the Frobenius norm) rank-one matrix to the original solution of the SPD relaxation. However, as you have pointed out, this does not guarantee that the resulting rank-one matrix satisfies any hard constraint from the original SDP anymore. It might satisfy your other constraints by a lucky coincidence, but this is not guaranteed. Oftentimes, since the SDP is a relaxation anyways, a lot of articles breeze over this issue of potentially losing feasibility after performing the rank-one projection!

One potential remedy is to compute the projection of the SDP-relaxed solution onto the intersection of the set of rank-one matrices with your original constraints (if they are required for your desired solution). However, this is not a straightforward task and requires solving another optimization problem in its own right.

  1. The other remedy is to (instead of solving an SDP relaxation) just directly parameterize the variables within a rank-one matrix and try to solve that optimization problem instead. By hard-coding the rank-one structure, you can sometimes circumvent the "nonconvexity" issues of the underlying optimization problem, leading to efficient solution techniques.
Zim
  • 4,623
  • Really thanks for the answer. Yes, as you mentioned, the first method is likely to break the constraints.

    For the potential remedy, I can get your point, but how to actually implement it? Could you offer me some related research?

    For the second problem, do you mean directly solve the non-convex problem by using somthing like yalmip?

     – tyrela Jun 12 '23 at 11:43
  • For the first method, I think you'd have to use some sort of heuristic (i.e., an unproven method) since I'm not actually sure about convergence guarantees because one of the sets is convex. But, to do it anyways, you'd minimize $|x-z|^2$ over $z\in C\cap R$, where $C$ is the set of your original constraints, and $R$ is the set of rank-one matrices. This could be done via the forward-backward or Douglas-Rachford algorithms in a product space, for instance. – Zim Jun 12 '23 at 19:14
  • For the second approach, it would be something like, instead of solving over a full matrix variable $x$ and requiring it to be in the set of rank-one matrices, you solve over two vectors $u,v$ and wherever $x$ appears in your SDP, just replace it with the outer product $uv^\top$. By solving for $u$ and $v$, you directly enforce the rank-one-ness of your solution. Sometimes this trick forces you to lose other nice properties (e.g., convexity) of the constraints or SDP objective, but (IIRC) not always. – Zim Jun 12 '23 at 19:16
  • Really thanks for your detailed answer! Now I totally get your points and I will see how to actually implement them. Meanwhile, I still have a minor question about the paper 1 that I cited in the post, and maybe you can enlighten me a little bit. After they solved the problem with SDR, they claimed "We use standard SDP methods to encourage low rank solutions [20]". But I don't really see how they actually encourage a low rank solution. The paper [20] they cited, are bascially some methods that minimizes the rank of a matrix as object and subjects to some convex constraints. – tyrela Jun 13 '23 at 08:00
  • Maybe they meant something like adding the trace(X) (X is the variable) in the object of the original problem formulation to minimize the rank of X as much as possible? Sorry I can't type all the words in the last comment. – tyrela Jun 13 '23 at 08:02
  • It looks like the convexification they're advocating here (in eq. (9) of [20] / reference 2) is the nuclear norm. This is a standard way to promote low-rank-ness of a solution while keeping your optimization problem convex. – Zim Jun 13 '23 at 08:14
  • So it's like adding the minimization of nuclear norm in the object of the problem formulation? – tyrela Jun 13 '23 at 08:38
  • the article is not sufficiently clear, but yes, that would be my guess. – Zim Jun 13 '23 at 09:27
  • Thank you very much! I will keep learning about this. – tyrela Jun 13 '23 at 11:16
  • Thank you for the useful answer :) if I may ask, are there any known results about relaxed problems whose SDP solution, has a rank > 1, but all of its corresponsing singular vectors are solutions to the non-relaxed problem? – Javier TG Nov 02 '23 at 02:08
  • 1
    @JavierTG that would be a super powerful result! I'm not aware of one, but I would not be surprised if some folks had figured out (at least some special instance) where this could occur. The first place I'd look in the literature would be searching the keywords "matrix recovery" and "hard thresholding singular values". – Zim Nov 16 '23 at 11:23
  • @Zim Thank you for the insight! Can you provide some references about the second approach? – GBmath Aug 23 '24 at 15:10
  • @goldboy sure, here is a review article which details several approaches along that line of work (see also its citations for further references): https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8811622

    This review is for general low-rank (rank one would be $r=1$ in their notation) and unfortunately, most of the time, the resulting "direct-parameterization" approach will yield a nonconvex problem.

     – Zim Aug 27 '24 at 15:17