5

I've been reading Boyd & Vandenberghe and it says that conic programming is a subclass of convex optimization. I haven't been able to find an example of a convex optimization problem that I cannot translate to an equivalent conic optimization problem (equivalent in the sense that I can find a solution to my original problem given a solution to the conic program). Do you have any examples?

If so, how can it be solved automatically? My understanding is that the best solvers out there such as SDPT3, MOSEK or Sedumi work with conic programs.

Rodrigo de Azevedo
  • 23,223
IOS_DEV
  • 541
  • The answer to your question depends on what cones you allow in the conic form representation. For instance, a problem involving a constraint such as $t\geq e^x$ may be convex, but not representable as a second-order cone problem. It only becomes conic representable when you add the exponential cone to your set of allowed cones. – Michal Adamaszek Oct 24 '22 at 10:04
  • How about training a logistic regression model? – littleO Oct 24 '22 at 10:53
  • @MichalAdamaszek assume we have all the convex cones. If we did, could we express any convex optimization problem? That is not so clear to me. – IOS_DEV Oct 24 '22 at 15:34
  • @littleO could you elaborate?? – IOS_DEV Oct 24 '22 at 15:35
  • 1
    @IOS_DEV Section 7.1 of Convex Optimization by Boyd and Vandenberghe contains a discussion of logistic regression. You'll see the optimization problem that we must solve when training a logistic regression model. Here's a link: https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf – littleO Oct 25 '22 at 20:54
  • A convex problem that's not formulated in the conic form could be solved with general nonlinear methods like interior point method or augmented Lagrangian method. Most of these methods have better convergence, etc. for convex problems. – Arnold Nightingale Jul 06 '23 at 20:09

2 Answers2

3

Let $C \subset \mathbb R^n$ and $f \colon \mathbb R^n \to \mathbb R$ be convex. We consider $$ \text{minimize} \; f(x) \; \text{s.t.} \; x \in C. $$ In order to simplify the objective, we introduce $D := (C \times \mathbb R) \cap \operatorname{epi} f$ and the problem is equivalent to $$ \text{minimize} \; t \; \text{s.t.} \; (x,t) \in D. $$ It remains to transform $D$ into a cone. To this end, we set $E := \operatorname{cl\,cone}( D \times \{1\})$ and one can check that $(x,t) \in D$ is equivalent to $(x,t,1) \in E$. Thus, our original problem is equivalent to the conic problem $$ \text{minimize} \; t \; \text{s.t.} \; (x,t,s) \in E\;\text{and}\; s = 1. $$

gerw
  • 33,373
2

Every convex function $f$ can be represented via a convex cone by defining $$K_f = \operatorname{cl} \{(u,t,x) \in \mathbb{R}_> \times \mathbb{R} \times \mathbb{R}^n;\ t \ge u f(x/u) \}.$$ Then $f(x) \le t$ if and only if $(1,t,x) \in K_f$.

Using this trick you can reformulate every convex optimization problem as a conic optimization problem. If $K_f$ turns out to be one of the standard cones, like the positive semidefinite cone, you're in luck. However, this will not be of any use if $f$ is some nasty function that generates some nasty convex cone $K_f$. For example, $f(x) = -\cos(x)$ is convex for $-\pi/2 \le x \le \pi/2$, but no solver implements this cone and I'd be really impressed if some managed to. You'd need the barrier function for its cone, and I would bet that no analytical expression for it exists.

Often you deal with cones that you don't have a solver for by representing them in terms of cones you do. For example, plenty of norm minimization problems can be reformulated in terms of the positive semidefinite cone. As you might expect, this is not always possible. There are plenty of cones that cannot be represented in terms of the usual ones (non-negative, positive semidefinite, second order, and exponential).

For example, in quantum mechanics one can define the convex cone of the (unnormalized) separable states, and one would really like to optimize over it. Recently it was proven, however, that this cone cannot be represented in terms of the positive semidefinite cone.

Mateus Araújo
  • 337