0

I need to show the following:

$f(X) = Tr (X^{-1})$ is convex on $ \text{dom} f = S^n_{++}$

I tried solving it by using the definition of convex function and first-order condition however I got stuck at the last part of the proof (I will open a separate question for that.)

I then stumbled upon this answer to the same problem which seemed very elegant. However, the author of the answer utilizes a "theorem" or property for which I cannot seem to find a reference, claiming (in the comments to the answer) that:

A twice differentiable function $f$ on an open subset $U$ of a vector space $V$ is convex iff $$ \frac{d^2}{dt^2} f(u+tv) \Big|_{t=0} \ge 0$$ for all $u \in U$, $v \in V$.

To me, this looks like a combination of verifying convexity by considering an arbitrary line $u+tv$ and second-order condition for checking the convexity of the function $g(t)=f(u+tv)$ since the second derivative with respect to $t$ must be $\ge 0$.

In the book Convex Optimization, by Boyd and Vandenberghe it says the following:

A function is convex if and only if it is convex when restricted to any line that intersects its domain. In other words, $f$ is convex if and only if for all $x \in \textbf{dom} f$ and all $v$, the function $g(t) = f(x+tv)$ is convex on its domain, $ \{t | x+tv \in \textbf{dom} f \}$.

Second order condition:

For a function on $\mathbb{R}$, the second-order condition reduces to $f''(x) \ge 0$ (and $\textbf{dom} f$ must be convex), which means that the derivative is non-decreasing.

The following remains unclear to me:

Why is it sufficient to check the second-order condition only at $t=0$? Shouldn't we prove that $g''(t) \ge 0$ on the whole domain $ \{t | x+tv \in \textbf{dom} f \}$ to prove $g$ is convex (which then implies $f$ is convex)

The author of the answer notes that it is ok to only evaluate at $t=0$ because:

The point is that other values of t (for which $u+tv \in U$) correspond to other choices for $u$ at $t=0$.

From what I understood this implies that we can do this because you can reach every element in the $\textbf{dom}f = U$ by setting the $t=0$. This seems like a trivial property if $u \in U$ and $t=0$ since that will always result with $u+tv \in U$. However this just "feels wrong" since we are not really considering lines through the points in the domain but individual points.

I would be grateful if somebody could clarify if I understood this correctly and if this "theorem" is true since it seems to significantly simplify verifying convexity for the example I'm trying to solve. Does anyone have any reference or proof of this "theorem"?

Note: Somebody else asked the author for proof or reference in the comments (admittedly not in the most polite manner) but it remains unanswered.

Additionally in the Convex Optimization book, I have found a similar example (Example image), which proves concavity for $f(X) = \text{log } \text{det} X$ using the same approach. However in this specific example $g''(t) \le 0$ for all $t$ and is not evaluated just at $t=0$.

Antun Đuraš
  • 1
  • 1
    Notice that $\frac{d^2}{dt^2} f(u+tv)\vert_{t=0} = \langle D^2f(u)v, v \rangle$ with the Hessian/matrix of second derivatives $D^2f$ (by the chain rule). Therefore checkingthe condition given by you for all directions $v$ and all $u \in U$ is equivalent to showing that the Hessian is positive semidefinite which is equivalent to convexity if the function is twice differentiable. – Lukas Aug 13 '24 at 14:54
  • Thank you @Lukas , that cleared it up for me. One more question, since in my example $f : R^{n \times n} \mapsto R$ we would end up with the second derivative w.r.t. X being tensor of $[ (n \times n) \times (n \times n) ]$ dimension. I assume the first term in $\langle D^2f(u)v, v \rangle$ is generalized matrix-vector multiplication $z = D^2f(u)v$ and then we have the inner product of $\langle z,v \rangle$ which in my example is Frobenius product as $z,v \in R^{n \times n}$. Do you by any means know any good reference to this type of generalized second-order condition for convexity? – Antun Đuraš Aug 17 '24 at 12:35
  • This is exactly what is meant here: $D^2f(u)v$ is then an $n \times n$-matrix and we can take its Frobenius product with $v$. – Lukas Aug 19 '24 at 14:39

0 Answers0