10

Let

$$A := \left\{ u v^T : u \in \mathbb{R}^m, v \in \mathbb{R}^n, \|u\|_2 = \|v\|_2 = 1 \right\}$$

I would like to show that

$$\textrm{conv}(A) = B_* := \left\{ X \in \mathbb{R}^{m \times n}: \|X\|_* = \textrm{Tr} \left( \sqrt{XX^T} \right) \le 1 \right\}$$

I have shown that $\textrm{conv}(A) \subset B_*$. However, I have been stuck proving the opposite inclusion $ B_* \subset \textrm{conv}(A)$. Does anyone have any tips on how to show this inclusion?

One idea that I have is to take the SVD of an arbitrary matrix $X \in B_*$ such that $X=U\Sigma V^T$ where the sum of the elements of the diagonal matrix $\Sigma \le 1$. We also know that $U,V^T$ are orthonormal square matrices. However, I'm not sure where to go from there.

Rodrigo de Azevedo
  • 23,223
guanton
  • 365
  • 4
    Assume $X$ has $k$ nonzero singular values. Then you can write $X$ as the sum of $k$ rank 1 matrices. Since the singular values sum to be less than or equal to 1, you have a convex combination of matrices in the rank 1 "ball". – Jürgen Sukumaran Dec 17 '20 at 01:23
  • 2
    I see, so you can write $X= \sum \lambda_k u_k v_k^T $ where the sum of the $\lambda_k$ terms is less than or equal to 1. However, what if the sum is strictly less than 1? I guess if $0 \in A$ then this would suffice since we can add $(1-\sum \lambda_k) 0$ – guanton Dec 17 '20 at 01:58
  • 3
    Of course $0$ is in $C(A)$ (with $C$ denoting convex hull) since for any $X=uv^T\in C(A)$ we have also $-X = -uv^T\in C(A)$ and then $0.5X + 0.5 (-X) = 0 \in C(A)$ by convexity. – Jürgen Sukumaran Dec 17 '20 at 02:45
  • Another way you can prove it is via the convex envelope. Taking the bi-conjugate of the rank function bounded by box constraints gives you the nuclear norm (same as $l_0$ "norm" with $l_{\infty}$ constraint in a vector space) – iarbel84 Dec 17 '20 at 08:26
  • Related: https://math.stackexchange.com/q/1099320/339790 and https://math.stackexchange.com/q/1914889/339790 – Rodrigo de Azevedo Dec 18 '20 at 01:01

1 Answers1

6

The claim is a consequence of the following key result (see Theorem 1 in Section 5.1.4 of Maryam Fazel's thesis [1], proved using conjugate functions in Section 5.1.5):

The convex envelope (the largest lower-bounding convex function) of the rank function $\text{Rank}(X)$ over the set $$S=\left\{ X \in \mathbb{R}^{m \times n}: \|X\| = \sqrt{\lambda_{\max}\left( XX^T \right)} \le 1 \right\}$$ is the nuclear norm $\|X\|_*$.

Above, $\|X\|$ is the spectral norm of $X$, also denoted by $\|X\|_2$ [2]. Note that the set of matrixes $X$ that satisfies $\text{Rank}(X)=1$ and $\|X\| \le 1$ includes the set $A$ considering $\|uv^T\|=\|u\|_2\|v\|_2$ [3]. Moreover, $\|X\|_* \le 1$ implies $\|X\| \le 1$ by $\|X\| \le \|X\|_F \le \|X\|_*$, thus the set of matrixes $X$ that satisfies $\|X\|_* \le \text{Rank}(X)=1$ and $\|X\| \le 1$ equals $B_*$. Indeed, $B_*$ is the smallest convex set including $A$ because otherwise $\|X\|_*$ cannot be the convex envelop of $\text{Rank}(X)$ over $S$ by definition.

Amir
  • 11,124