1

Let $x \in \mathbb{R}^n$ and $P$ a $n \times n$ positive definite symmetrix matrix. It is known that $\max_{x|~x^Tx\leq 1} x^TPx=\lambda_{\text{max}}(P)$, where $\lambda_{\text{max}}(P)$ is the largest eigenvalue of $P$.

Now split the vector $x$ in two parts as $x=\begin{bmatrix}x_1\\\tilde{x}\end{bmatrix}$, where $x_1 \in \mathbb{R}^{n_1}$ and $\tilde{x}$ is constant. Consider the following optimization problem $$y=\text{arg}\max_{x|~{x_1}^Tx_1\leq 1} x^TPx$$ that is, the subvector $x_1$ is required to be in a sphere, while the subvector $\tilde{x}$ is known and constant. The only decision variable is then $x_1$. Can I find the solution analytically?

pulosky
  • 675

1 Answers1

1

You can divide $P$ into blocks:

$$P = \begin{bmatrix} P_1 & Q \\ Q^t & R \end{bmatrix}$$

Then we get

$$\begin{align*} x^t P x &= x_1^t P_1 x_1 + x_1^t Q \bar x + \bar x^tQ^tx_1 + \bar x^t R \bar x\\ & = x_1^t (P_1 x_1 + 2Q\bar x)+ \underbrace{\bar x^t R\bar x^t}_{const}\end{align*}$$

As $P$ is positive definite, all it is leading principal minors are positive and therefore $P_1$ is SPD as well. Furthermore we see that

$$\arg\max_{x_1: x_1^tx_1 \leq 1} x^t P x = \arg\max_{x_1: x_1^tx_1 \leq 1} x_1^t(P_1x_1 + 2Q\bar x)$$

Can you continue from here?

Edit: To use the same tool from above again, we have to complete the square, this means bringing that expression to the form $(x_1-a)^tP_1 (x_1-a) + c$. By expanding we get

$$(x_1-a)^tP_1 (x_1-a) + c = x_1^t P_1 x_1 - 2x_1^t P_1 a + a^t P_1 a+c$$

So we see $P_1 a = Q\bar x$ or $a = P_1^{-1}Q\bar x$ and $c = -a^t P_1 a$. So for $y := x_1 - a$ we are looking now for the $y$ maximizing $y^tP_1y$. Make sure to also express the condition $x_1^tx_1 \leq 1$ in terms of $y$.

flawr
  • 16,931
  • Thanks! I had actually come to your reformulation. Now it is a QCQP, but I can't see how the analytical solution is obtained. Seems to me that the $2Q\overline{x}$ term ruins the max eigenvalue result. – pulosky Sep 22 '17 at 09:49
  • You can eliminate this term by "completing the square", i.e. transforming it to an expression of the form $(x_1-a)^tP_1 (x_1-a) + const$. After that, you can define the variable transformation $y = x_1 -a$ and continue. – flawr Sep 22 '17 at 09:52
  • Thanks. I can get to: $$\text{arg}\max_{||y-P_1^{-1}Q\overline{x}|| \leq 1} y^TP_1 y - \overline{x}^TQ^TP^{-T}Q \overline{x} $$. However, $y$ is now constrained in a shifted sphere. How could I continue? – pulosky Sep 22 '17 at 10:03
  • Opened a new question for the reformulated problem: https://math.stackexchange.com/questions/2440218/maximization-over-shifted-sphere?noredirect=1#comment5040857_2440218 – pulosky Sep 22 '17 at 10:30