Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [convex-optimization]

A convex optimization problem consists of either minimizing a convex objective or maximizing a concave objective over a convex feasible region.

Convex Optimization is a special case of mathematical optimization where the feasible region is convex and the objective is to either minimize a convex function or maximize a concave function. Linear Programming is a special case. Convex Optimization problems as a class are easier to solve numerically than general mathematical optimization problems.

The following problems are all convex minimization problems, or can be transformed into convex minimizations problems via a change of variables:

  • Least squares
  • Linear programming
  • Convex quadratic minimization with linear constraints
  • Quadratic minimization with convex quadratic constraints
  • Conic optimization
  • Geometric programming
  • Second order cone programming
  • Semidefinite programming
  • Entropy maximization with appropriate constraints
7685 questions
312
votes
8 answers

Please explain the intuition behind the dual problem in optimization.

I've studied convex optimization pretty carefully, but don't feel that I have yet "grokked" the dual problem. Here are some questions I would like to understand more deeply/clearly/simply: How would somebody think of the dual problem? What…
littleO
  • 54,048
188
votes
12 answers

The median minimizes the sum of absolute deviations (the $ {\ell}_{1} $ norm)

Suppose we have a set $S$ of real numbers. Show that $$\sum_{s\in S}|s-x| $$ is minimal if $x$ is equal to the median. This is a sample exam question of one of the exams that I need to take and I don't know how to proceed.
hattenn
  • 2,287
72
votes
4 answers

Difference between supremum and maximum

Referring to this lecture , I want to know what is the difference between supremum and maximum. It looks same as far as the lecture is concerned when it explains pointwise supremum and pointwise maximum
user31820
  • 1,033
71
votes
2 answers

What is the difference between minimum and infimum?

What is the difference between minimum and infimum? I have a great confusion about this.
Manoj
  • 1,827
63
votes
5 answers

Geometric intuition of conjugate function

I am looking for a geometric and intuitive explanation of the conjugate function and how it maps to the below analytical formula. $$ f^*(y)= \sup_{x \in \operatorname{dom} f } (y^Tx-f(x))$$
Abhishek Bhatia
  • 1,758
  • 7
  • 23
  • 38
49
votes
8 answers

How do you prove that $\{ Ax \mid x \geq 0 \}$ is closed?

Let $A$ be a real $m \times n$ matrix. How do you prove that $\{ Ax \mid x \geq 0, x \in \mathbb R^n \}$ is closed (as in, contains all its limit points)? The inequality $x \geq 0$ is interpreted component-wise. This fact is used in some proofs…
littleO
  • 54,048
34
votes
1 answer

Convexity of the product of two functions in higher dimensions

Exercise 3.32 page 119 of Convex Optimization is concerned with the proof that if $f:\mathbb{R}\rightarrow\mathbb{R}:x\mapsto f(x)$ and $g:\mathbb{R}\rightarrow\mathbb{R}:x\mapsto g(x)$ are both convex, nondecreasing (or nonincreasing) and positive,…
Anthony Labarre
  • 888
32
votes
4 answers

KKT and Slater's condition

I was studying Stephen Boyd's textbook and got confused in the KKT part. The book says the following: For any convex optimization problem with differentiable objective and constraint function, any points that satisfy the KKT conditions are primal…
DSKim
  • 1,177
  • 4
  • 14
  • 18
28
votes
4 answers

How the dual LP solves the primal LP

When I heard someone discussing LP the other day, I heard him say, "Well, we could just solve the dual." I know that both the primal LP and its dual must have the same optimal objective value (assuming both are feasible and bounded). I also…
user
  • 413
27
votes
2 answers

What is the intuition behind Slater's condition in optimization? (And other constraint qualifications.)

I would like to "grok" Slater's condition and other constraint qualification conditions in optimization. Slater's condition is only one of many different constraint qualifications in the optimization literature. Which one is the most fundamental? …
eternalGoldenBraid
  • 2,209
26
votes
2 answers

Second derivative positive $\implies$ convex

In proof of the following theorem; If $f$ has a second derivative that is non-negative (positive) over an interval then $f$ is convex (strictly convex). $f$ is in real number space., the book I refer, uses Taylor series expansion but disregards…
triomphe
  • 3,948
24
votes
2 answers

How to physically interpret conjugate functions?

If given a convex function $f: \mathbb{R} \to \mathbb{R}$, then the conjugate function $f^*$ is defined as $$f^*(s) = \sup_{t \in \mathbb{R}} (st-f(t))$$ Now i want to understand what is the physical interpretation of this conjugate function? What…
PAMG
  • 4,642
23
votes
2 answers

Are Fenchel and Lagrangian duals equivalent?

Consider the Fenchel dual and the Lagrangian dual. Are these duals equivalent? In other words, is using one of the these duals (say for solving an optimization), would give the same answer as using the other one? I think the answer is no, but I am…
Daniel
  • 2,760
23
votes
2 answers

Convex analysis books and self study.

I have taken some courses in Convex optimization. Now I would like to know a little bit more about the pure mathematical side. Is there any good books in convex analysis? I have read and worked with Boyds Convex Optimization book. Is there any video…
Johan
  • 4,072
22
votes
3 answers

Pointwise infimum of affine functions is concave

So I was just starting on convex optimization and was having a slightly hard time visualizing the lagrangian being always concave because it is the pointwise infimum of a family of affine functions. Can anyone help explain this? I've googled…
Achint
  • 351
1
2 3
…
99 100