1

Consider the optimization problem

$$\min c^Tx,$$ such that $$Ax=0,\\ x^Tx\leq1.$$ where $$c^T[I-A^T(AA^T)^{-1}A]c > 0,$$ and $A$ is an $m\times n$ matrix of rank $m$.

My attempt:

I try to solve it using Lagrange Method, and I can write down the first-order sufficient conditions: $$c+A^T\lambda + 2\mu x = 0, \\ Ax = 0, \\ \mu(x^Tx-1)\leq 0, \\ \mu \geq 0.$$ Then, we can first assume that the inequality constraint is active, which means that $x^Tx=1$, there is $$c+A^T\lambda + 2\mu x = 0, \\ Ax = 0, \\ x^Tx-1 = 0.$$

Then, I have no idea how to solve this equation system.

Palmer
  • 119

1 Answers1

2

Let $N$ be a matrix where its columns consists of basis of nullspace of $A$.

That is we can define a vector $y$ such that $Ax = 0$ can be rewritten as $x=Ny$. That is rather than solving for $x$, we can solve for $y$.

$x^Tx=1$ is equivalent to $y^TN^TNy=1$.

Now, we are trying to solve for

$$\min (c^TN)y$$

subject to $$y^T(N^TN)y=1$$

which is equivalent to $$\max (-c^TN)y$$

subject to $$y^T(N^TN)y=1.$$

It is an optimization problem over an ellipsoid which has been addressed here

The optimal $y$ is

$$y^*=-\frac1{\sqrt{(c^TN)(N^TN)^{-1} (N^Tc)}} (N^TN)^{-1}N^Tc$$

and we can compute $x^*=Ny^*$.

Siong Thye Goh
  • 153,832
  • $x^{} = N y^{}$ is the normalized orthogonal projection of $-c$ onto the kernel of A (range N). $y$ can be calculated efficiently in most programming languages -- order $O(n^3)$ in products (and sums) of reals. That is, $x^{}=-\alpha (I-A^{\dagger} A)c=-\alpha(c-A^{\dagger} Ac) $, for some $\alpha>0$ such that $\dfrac{x^{}}{|x^{}|}.$ Here, $A^{\dagger} A c $ is the least square solution of the system $Ax=Ac$ with minimum norm (not necessarily $d=c$ achieves minimum norm). In Julia Programming Language, I would do $$x^{}= - \dfrac{c-(A \setminus Ac)}{|c-(A \setminus Ac)|}.$$ – R. W. Prado Aug 16 '23 at 07:07