4

I'm confused about sufficiency of KKT conditions for optimality.

In http://www.stat.cmu.edu/~ryantibs/convexopt/scribes/kkt-scribed.pdf, page 3, Section 12.1.3 it is stated that

For any optimization problem, if $x^\star$ and $(u^\star,v^\star)$ (optimal lagrange multipliers) satisfy KKT conditions for the problem, then satisfying those KKT conditions is sufficient to imply that $x^\star$ and $(u^\star,v^\star)$ are the optimal solutions for the primal and it’s dual. This statement is equivalent to saying satisfying KKT conditions is always sufficient for optimality.

The argument is based on the fact that if $(x^\star,u^\star,v^\star)$ is a KKT point, then it must hold $g(u^\star,v^\star)=f(x^\star)$ because of primal/dual feasibility and complementary slackness. Hence, I would deduce that if there is a KKT point, then strong duality holds and the global optimum is found.

However, I thought that KKT conditions are only necessary for optimality under the assumption that strong duality holds - thus my confusion.

Could someone clarify? Also, supposing that strong duality does not hold, what can we say about KKT points? Thanks.

glS
  • 7,963
pulosky
  • 675
  • related: https://math.stackexchange.com/q/2214851/173147, https://math.stackexchange.com/q/3616646/173147 – glS Apr 03 '24 at 14:48

1 Answers1

5

It is not well written there. KKT condition is, in general, only necessary condition. It is in a way similar to the "gradient is zero" for unconstrained case. It is no way sufficient, unless the problem is convex. I guess the text implicitly assumes convexity. Otherwise, (12.14) is not true that stationarity implies minimum. However, if the problem is convex then it is true that the KKT condition implies saddle point and, hence, strong duality.

P.S. The word "convexopt" in the link reveals that the text is likely a part of a convex optimization course. It explains "sloppiness" in not mentioning convexity.

A.Γ.
  • 30,381
  • Thanks, yes, with hindsight you are probably right that convexity is assumed. Do KKT points reveal anything interesting for general non-convex problems for which strong duality does not hold? – pulosky May 14 '18 at 14:55
  • @pulosky Even if the strong duality hold, a KKT point may have nothing to do with minimum. There can be many KKT points. – A.Γ. May 14 '18 at 15:04
  • isn't this in contradiction with eg https://math.stackexchange.com/a/2513724/173147? For the classic example $\min x$ subject to $x^2\le0$, the solution is $x=0$ but it's not a KKT point. Thus KKT are not necessary in general. – glS Apr 03 '24 at 08:15
  • @glS The precise formulation is that the KKT condition is necessary under some mild assumptions: differentiability and a constraint qualification condition. The latter may take slightly different shapes, but the idea is that the constraints at the minimal point should behave well, for example, the gradients are linearly independent. For the constraint $x^2\le 0$, the gradient (actually, simply the derivative) is zero at the minimum point $x=0$, thus, dependent. The constraint qualification is not satisfied. Such cases are relatively rarу in practice, but are a headache in theory. – A.Γ. Apr 03 '24 at 14:37