2

Can we prove that $\mbox{trace}({\bf A} ({\bf P}+{\bf Q})^{-1} {\bf A}^T)$ is a jointly convex function of positive variables $[q_1,q_i,...,q_N]$, where ${\bf Q}=\mbox{diag}(q_1,...,q_N)$, $q_i>0 , \forall i$, and ${\bf P}$ is a positive definite matrix.

Bamboo
  • 2,295
Alireza
  • 553

2 Answers2

4

The epigraph of this function is $$\mathop{\textrm{epi}} f(P,Q) = \left\{ (P,Q,z) \,\middle|\, P+Q\succ 0,~\mathop{\textrm{Tr}}(A(P+Q)^{-1}A^T)\leq z\right\}$$ This is equivalent to $$\mathop{\textrm{epi}} f(P,Q) = \left\{ (P,Q,z) \,\middle|\, \exists Z ~~ P+Q\succ 0,~\begin{bmatrix} Z & A^T \\ A & P+Q \end{bmatrix} \succeq 0, ~ \mathop{\textrm{Tr}}(Z) \leq z \right\}$$ The epigraph is the intersection of linear matrix inequalities and a linear inequality, composed with the projection $(P,Q,Z,z)\rightarrow (P,Q,z)$, so it is convex.

Michael Grant
  • 20,110
3

Notice that the inverse of a symmetric positive definite matrix is convex (cf Is inverse matrix convex?) w.r.t. to the cone of symmetric positive semidefinite matrices $S^n_+$.

Edited Plain version:

For $t\in (0,1)$ and symmetric positive definite $X,\tilde X$ it follows the matrix $$ (1-t) X^{-1} + t\tilde X^{-1} - ((1-t) X + t \tilde X)^{-1} $$ is positive semidefinite. In particular, $$ A[(1-t) X^{-1} + t\tilde X^{-1} - ((1-t) X + t \tilde X)^{-1}]A^T $$ is positive semidefinite. Thus, $$ \operatorname{trace}(A[(1-t) X^{-1} + t\tilde X^{-1} - ((1-t) X + t \tilde X)^{-1}]A^T) \ge 0 $$ and \begin{align} \operatorname{trace}(A((1-t) X + t \tilde X)^{-1}A^T) &\le \operatorname{trace}(A[(1-t) X^{-1} + t\tilde X^{-1}]A^T) \\ &= (1-t)\operatorname{trace}(AX^{-1}A^T) + t\operatorname{trace}(A\tilde X^{-1}A^T). \end{align} Thus, $X\mapsto \operatorname{trace}(AX^{-1}A^T)$ is convex. Since, $(P,Q)\mapsto P+Q$ is affine, $(P,Q)\mapsto \operatorname{trace}(A(P+Q)^{-1}A^T)$ is also convex.

Composition rule with generalized inequality and monotonicity version:

Notice that $\operatorname{trace}(AXA^T)$ is convex (in fact linear) and monotone increasing in $X$ w.r.t. $S^n_+$. Thus, the composition $(P,Q)\mapsto \operatorname{trace}(A(P+Q)^{-1}A^T)$ is convex.

Community
  • 1
user251257
  • 9,417
  • thank you for the reply, the thing is that $Q$ is diagonal, you have proved that the trace is convex with respect to $Q$ , can we conclude that it is convex in $q$ too? – Alireza Sep 17 '15 at 22:10
  • @Alireza: if a function is convex, it is also convex on a convex subset – user251257 Sep 17 '15 at 22:11
  • I actually don't think this is a valid approach, at least not without further development. The convexity of the inverse you have linked to is defined with respect to the generalized inequality induced by the semidefinite cone. How do we know that the composition of a function that is convex in one cone and a function that is increasing and convex on another cone is convex? That's not proven in typical convex optimization texts, to be sure. – Michael Grant Sep 18 '15 at 02:04
  • @MichaelGrant, I wrote that it is monotone with respect to $S^n_+$? – user251257 Sep 18 '15 at 02:06
  • Yes, but I don't think that matters. Again we're talking about two different kinds of convexity here. This is not the standard Boyd & Vandenberghe composition rule we're talking about. – Michael Grant Sep 18 '15 at 02:07
  • @MichaelGrant, I can't follow: We have $((1-t)P + tQ)^{-1} \succeq (1-t)P^{-1} + tQ^{-1}$ and $\langle M, X \rangle \ge 0$ for every $M,X\succeq 0$. I don't really get your point – user251257 Sep 18 '15 at 02:09
  • That is not the standard definition of convexity. That is convexity defined with respect to the semidefinite cone. You cannot apply the various composition rules in standard convex analysis texts to this generalized notion of convexity---at least not without additional work to prove they apply. Standard convexity is defined with respect to $\mathbb{R}_+$ only. – Michael Grant Sep 18 '15 at 02:11
  • @MichaelGrant: I don't need a general composition rule (despite the fact it is mention in Boyd's Convex Optimization). It follows directly from these both inequality. After all $\langle M, \cdot \rangle$ is linear. – user251257 Sep 18 '15 at 02:15
  • I disagree. That is to say, I don't think it is common knowledge that if $F(x)$ is convex on the semidefinite cone, $\mathop{\textrm{Tr}}(F(x))$ is convex by the standard definition. It's not hard to prove but it must nevertheless be proven, or a citation offered. – Michael Grant Sep 18 '15 at 02:22
  • Well, at least, this comment thread has offered enough information to complete the matter :-) – Michael Grant Sep 18 '15 at 02:24
  • @MichaelGrant: I added a vanilla version. However, after writing it, I had to admit that the epigraph version is really elegant. – user251257 Sep 18 '15 at 02:39
  • Great! Thanks for the kinds words but truthfully yours is the straightforward answer, especially now with the further development. – Michael Grant Sep 18 '15 at 03:09
  • @MichaelGrant Thank you very much guys. I have another question related, suppose I want to maximize this function with respect to $q_i$'s with a equality constraint like $\prod q_i=a$, can I use KKT conditions for this? – Alireza Sep 21 '15 at 21:54
  • @user251257 Thank you very much guys. I have another question related, suppose I want to maximize this function with respect to qi's with a equality constraint like ∏qi=a, can I use KKT conditions for this? – Alireza – Alireza Sep 21 '15 at 21:55
  • @Alireza you should post an other question. In short: you could. However, it is not convex. So it gets nasty. – user251257 Sep 22 '15 at 11:38