22

So I was just starting on convex optimization and was having a slightly hard time visualizing the lagrangian being always concave because it is the pointwise infimum of a family of affine functions.

Can anyone help explain this? I've googled extensively but most places just state this without elaboration or examples.

Thanks.

Amzoti
  • 56,629
Achint
  • 351
  • 11
    Affine functions are themselves concave (and convex). The pointwise infimum of concave functions is concave. You will probably find more for the equivalent: The pointwise supremum of convex functions is convex. A function is convex if and only if its epigraph is convex, and the epigraph of a pointwise supremum is the intersection of the epigraphs. – Daniel Fischer Oct 05 '13 at 18:59
  • Hi, thank you, there was more lucid literature from that point of view. – Achint Oct 08 '13 at 01:08
  • Just a quick comment that the Lagrangian per se is not always concave as stated by OP (I have edited your post) - I think OP is referring to the dual function of the Lagrangian that takes the pointwise infimum. That is concave, regardless of whether the original objective function is convex, concave, or neither. – Samuel Low Feb 03 '24 at 22:10

3 Answers3

16

Daniel Fischer gave a transparent explanation in terms of epigraphs $\{(x,y): y\ge f(x)\}$:

A function is convex if and only if its epigraph is convex, and the epigraph of a pointwise supremum is the intersection of the epigraphs. [Hence,] the pointwise supremum of convex functions is convex.

One can similarly argue from concavity, using the sets $\{(x,y): y\le f(x)\}$: this set is convex if and only if $f$ is concave. Taking infimum of functions results in taking the intersection of such sets.

user103402
  • 2,753
15

See picture below for some intuition on why that may be (bold line is the pointwise infimum of the four affine functions). Similarly, you can conclude that the

Pointwise supremum of the set of affine functions is convex

enter image description here

Tomasz Bartkowiak
  • 1,050
2

The point-wise maximum of a set of convex functions is convex. $$g(x) = \max_i f_i(x) $$ The corresponding epigraph $$\{(x,t) | t > \max_i f_i(x)\}$$ is convex which could also be visualized as the insertion of a family of convex spaces, $$\cap_i \{(x ,t)| t > f_i(x)\}$$

The lagrangian could be rewritten as the negative of the supremum, $$lagrangian(x) = \min_i f_i(x) = - \max_i -f_i(x)$$

fjqoebfpq
  • 21