5

At the beginning of chapter 9 of Boyd & Vandenberghe's Convex Optimization, about unconstrained minimization, it is said:

The starting point $x^{(0)}$ for a method must lie in $\mathbf{dom} f$, and in addition the sublevel set $$S = \{x \in \mbox{dom} \, f \mid f(x) \leq f(x^{(0)}) \}$$ must be closed. This is satisfied for all $x \in \mathbf{dom} f$ if the function is closed.

In other paper I've found an assumption, that a function $f$ must be convex, proper and closed in order that: minimize f is solvable. I understand why are the first two assumptions needed, but why is it important for the function to be closed so that problem is solvable? I would appreciate any explanation for the intuition (and what does this mean for the epigrpah for the objective function).

Rodrigo de Azevedo
  • 23,223
user402838
  • 51
  • I don't have an answer for you but here's a question: can you think of a couple of functions that are convex and don't satisfy this closedness property? If you can, edit your question and include them---and perhaps that will help spark a discussion about why the methods in Chapter 9 won't work with them. – Michael Grant Dec 30 '16 at 18:12
  • 1
    On a different note: Convex, proper and closed does not guarantee existence of a minimize as the exponential function shows. As for the closedness: Think about function where the minimum is at the boundary of domain and then remove that point from the domain. – Dirk Dec 30 '16 at 19:48
  • @MichaelGrant : I have not found any examples of such functions it this book ,so that is the reason why I was asking. I could not see why is this property so important. – user402838 Jan 01 '17 at 17:29
  • 1
    @Dirk Actually in Boyds Paper for ADMM, – user402838 Jan 01 '17 at 17:31
  • @Dirk for convergence he makes this assumption " The (extended-real-valued) functions $f : R^n → R ∪ {+ \infty}$ and $g : R^m → R ∪ {+ \infty}$ are closed, proper, and convex. [...Assumption 1] implies that the subproblems arising in the x-update (3.2) and z-update (3.3) are solvable ($ 'textbf{this 2 updates are just unconstrained minimzation problems}$), i.e., there exist x and z, not necessarily unique (without further assumptions on A and B), that minimize the augmented Lagrangian." link Page 16 – user402838 Jan 01 '17 at 17:36
  • @armincvx This assumption has been proven not to hold: https://arxiv.org/abs/1507.02051, crazy gravedig. – ms_ Apr 16 '21 at 20:46

2 Answers2

3

Here's an example of what can go wrong. Consider the domain

$D=\left\{x \in R^{2} | x_{2} > 0 \right\} \cup (0,0)$

Note that $D$ is convex.

Let

$f(x)=x_{2}$

Here, $f$ is convex on $D$, and the unique minimum of $f$ over $D$ is at the origin. However, the level sets of $f$ are not closed.

Now, apply the method of gradient descent to $f$, starting at the point $(1,1)$. With appropriate step lengths, you can get the method to converge to $(1,0)$, which is outside of $D$ and nowhere near the minimizer at the origin.

Brian Borchers
  • 11,308
  • Thank you for your answer. This example makes it quite obvious why this property is important. But I have been wondering is there more behind this assumption then just to make sure the boundary points of the set are feasible, so pathology cases like your example can be avoided. – user402838 Jan 01 '17 at 22:52
  • I'd be interested to see an example in which violation of the assumption of closed level sets lead to failure in some significantly different way than in my example. I'm not aware of any. – Brian Borchers Jan 02 '17 at 00:16
  • Since solving unconstrained problems with descent methods are basically iterative methods for solving the optimality condition $ \nabla f(x) = 0$. I have been wondering what conclusions we can draw from a function being closed for solving the optimaltz conditions. But on the other hand, if a functions is closed, its derivative does not have to be closed, thats why I have been confused about this assumption. – user402838 Jan 02 '17 at 13:26
0

Another example is minimizing $f(x) = 1/x$. The minimum exists, and is 0, but the minimizer does not exist, and most standard methods behave oddly on this problem. The function's epigraph is

$$ \mathbf{epi}~f = \{(x,f(x)): x\in\mathbb R\} $$ and does not include the point $(x,0)$ for any value of $x$ (but the sequence $(n,f(n))$ is in the epigraph and approaches the x axis). Therefore the epigraph is not closed, and the function is not closed.

Y. S.
  • 1,876