Constrained convex optimization

Question

Solve

Maximize $f(x)=c^Tx$

subject to $x^TQx \leq 1$

where $Q$ is a positive definite matrix.

what is the solution if the objective function is to be minimized ?

Answer 1

Write $Q=P^TDP$, where $D={\rm diag}(\lambda_1,\dots,\lambda_n)$ and $P^T=P^{-1}$. Since all the $\lambda_i$ are positive, the matrix $D^{1/2}={\rm diag}(\sqrt{\lambda_1},\dots,\sqrt{\lambda_n})$ is well defined and satisfies $(P^TD^{1/2}P)^2=Q$. So we can use the (well-known) notation $$ Q^{1/2}=P^TD^{1/2}P. $$ Then $$ \qquad x^TQx = x^TQ^{1/2}Q^{1/2}x = \Vert Q^{1/2}x\Vert_2^2.\qquad(1) $$ Therefore, for $y=Q^{1/2}x$ and $d=Q^{-1/2}c$, the problem can be restated as $$ \begin{matrix} \rm maximize &d^Ty\\ \rm subject\ to &\Vert y\Vert_2^2\le 1, \end{matrix} $$ which attains its optimal point at $d/\Vert d\Vert_2$ by virtue of the Cauchy-Schwartz inequality $|u^Tv|\le\Vert u\Vert_2\Vert v\Vert_2$ (the inequality implies that $d^Ty\le\Vert d\Vert_2$ if $\Vert y\Vert_2\le 1$, with equality for $y=d/\Vert d\Vert_2$). Thus, the solution to the original problem is $$ \Vert d\Vert_2 = \Vert Q^{-1/2}c\Vert_2, $$ attained at the optimal point $$ y^* = \frac{Q^{-1/2}c}{\Vert Q^{-1/2}c\Vert_2}, $$ which can be re-written as $$ x^* = \frac{Q^{-1}c}{\sqrt{c^TQ^{-1}c}} $$ because $\Vert Q^{-1/2}c\Vert_2=\sqrt{c^TQ^{-1/2}Q^{-1/2}c}$.

Note: In the case of minimization the optimal point is $-x^*$ and the optimal value is $-\Vert d\Vert_2$. To see this simply use the other end of the Cauchy-Schwartz inequality, which implies $d^Ty\ge-\Vert d\Vert_2$ if $\Vert y\Vert_2\le1$, with equality for $y=-d/\Vert d\Vert_2$.