Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [quadratic-programming]

Questions on quadratic programming, the optimization of a quadratic objective function subject to affine constraints.

Quadratic Programming (QP) is a special case of where a quadratic function (called the objective function) of one or several variables is optimised subject to only affine constraints. Compared to (LP), only the objective function is different.

For $n$ variables and $m$ constraints, the objective is find an $n\times1$ vector $\bf x$ under the following conditions. \begin{align}\min\quad&\frac12{\bf x}^\top Q{\bf x}+{\bf c}^\top{\bf x}\\\text{s.t.}\quad&A{\bf x}\preceq{\bf b}\end{align} where, in the real numbers,

  • $\bf c$ is an $n\times1$ vector

  • $\bf b$ is an $m\times1$ vector

  • $Q$ is an $n\times n$ symmetric matrix

  • $A$ is an $m\times n$ matrix.

Common methods to solve them include the augmented Lagrangian and conjugate gradients.

Reference: Wikipedia - Quadratic Programming

788 questions
29
votes
4 answers

How does the SVD solve the least squares problem?

How do I prove that the least-squares solution for $$\text{minimize} \quad \|Ax-b\|_2$$ is $A^{+} b$, where $A^{+}$ is the pseudoinverse of $A$?
Elnaz
  • 669
20
votes
5 answers

Maximize $x_1x_2+x_2x_3+\cdots+x_nx_1$

Let $x_1,x_2,\ldots,x_n$ be $n$ non-negative numbers ($n>2$) with a fixed sum $S$. What is the maximum of $x_1 x_2 + x_2 x_3 + \dots + x_n x_1$?
aditsu quit because SE is EVIL
  • 711
17
votes
2 answers

Are "constrained linear least squares" and "quadratic programming" the same thing?

A Quadratic Programming problem is to minimize: $f(\mathbf{x}) = \tfrac{1}{2} \mathbf{x}^T Q\mathbf{x} + \mathbf{c}^T \mathbf{x}$ subject to $A\mathbf{x} \leq \mathbf b$; $C\mathbf{x} = \mathbf d$; and $ \mathbf{s} \leq \mathbf{x} \leq \mathbf t$…
Meekohi
  • 273
13
votes
2 answers

Does gradient descent converge to a minimum-norm solution in least-squares problems?

Consider running gradient descent (GD) on the following optimization problem: $$\arg\min_{\mathbf x \in \mathbb R^n} \| A\mathbf x-\mathbf b \|_2^2$$ where $\mathbf b$ lies in the column space of $A$, and the columns of $A$ are not linearly…
syeh_106
  • 3,233
11
votes
2 answers

Linear programming with one quadratic equality constraint

I have a problem that can be formulated as a linear program with one quadratic equality constraint: where variable $x$ is an $n$-dimensional vector and $H$ is a positive semidefinite $n \times n$ matrix. I know this optimization problem can always…
user123346
  • 111
10
votes
3 answers

How do you minimize "hinge-loss"?

A lot of material on the web regarding Loss functions talk about "minimizing the Hinge Loss". However, nobody actually explains it, or at least gives some example. The best material I found is here from Columbia, and I include some snippets from it…
CodyBugstein
  • 1,650
10
votes
2 answers

Analog of Simplex Method for Quadratic Programming

It happened so I need to work with an algorithm for solving QP problems. The main issue here is that I can't find any references to this algorithms (at least no references in sources in english). There is something on Wikipedia, but rather…
Artem Pianykh
  • 211
10
votes
1 answer

Writing a convex quadratic program (QP) as a semidefinite program (SDP)

Given a convex quadratic program (QP) $$\begin{array}{ll} \underset{x}{\text{minimize}} & \mathrm x^\top \mathrm Q \, \mathrm x + \mathrm r^{\top} \mathrm x + s\\ \text{subject to} & \mathrm A \mathrm x \leq \mathrm b\end{array}$$ how can one write…
Rodrigo de Azevedo
  • 23,223
8
votes
2 answers

What Numerical Methods Are Known to Solve $ {L}_{1} $ Regularized Quadratic Programming Problems?

What numerical methods are suitable to solve the following problem $$\min_x \tfrac{1}{2}x^T A x + b^Tx + \lambda ||x||_1$$ where $x,b\in\mathbf{R}^n$, and $A\in \mathbf{R}^{n\times n}$ is positive definite, and $\lambda\in\mathbf{R}$ is…
Chris Taylor
  • 29,755
8
votes
2 answers

Quadratic function must be positive definite to have a unique minimum

Let $$V(x):=a+b^{T}x+\frac{1}{2}x^{T}Cx$$ for some $a \in \mathbb{R}$, $b \in \mathbb{R}^{n}$, $C \in \mathbb{R}^{nxn}$ that for $V$ to have a strict unique minimum it is imperative that $C>0$. I have attempted to solve this multiple times and I…
TorqueNoFriction
  • 165
8
votes
1 answer

Conditions for no duality gap in quadratic programming?

Assume $Q \in \mathbb{R}^{n\times n}$, and $b,c,d \in \mathbb{R}^n$. A quadratic programming problem is: $$ \min_{x \in \mathbb{R}^{n}} \tfrac{1}{2} x^T Q x + c^T x,$$ subject to $A x \leq b, E x = d$. I was wondering what are some sufficient…
Tim
  • 49,162
8
votes
2 answers

Projection of a point onto a convex polyhedra

Let $x_0 \in \mathbb{R}^n$ and $A \in \mathbb{R}^{n \times n}$ be given. Define the set $S$ as $$ S \triangleq \{x \in \mathbb{R}^n: A x \leq 1\}. $$ I want to compute the projection of $x_0$ onto $S$, i.e. the closest point to $x_0$ in $S$. Is…
pikachuchameleon
  • 2,889
8
votes
2 answers

Converting from QP to SOCP

I want to convert the following problem into SOCP form: $minimize \quad$ $x^TAx+a^Tx$: $subject$ $to \quad$ $Bx \leq b$ The approach I am taking is introducing new variables, $u$ and $v$, such that: $u=x^TAx$ and $v=a^Tx$. Then, our problem…
Kristada673
  • 565
8
votes
3 answers

Least squares Problem with Non Negativity Constraints

Let $\mathbf{x}=[x_1,\ldots,x_K]$. I have the following optimization problem: \begin{array}{rl} \min \limits_{\mathbf{x}} & \| \mathbf{Ax}-\mathbf{b} \|^2 \\ \mbox{s.t.} & x_k\ge 0, \forall k \end{array} Please I need your help to solve this…
tam
  • 571
7
votes
3 answers

Solving box-constrained least-squares

I have a linear least squares problem with linear constraints: $$\min_x \| A x - b \|^2 \quad\text{subject to}\quad k_1 \leq x_i \leq k_2$$ Should quadratic programming be used here? If so, what would the formulation be?
Jak
  • 71
1
2 3
…
52 53