Matrix inner product, and operator and trace norm inequality

Question

I have trouble proving the following inequality.

Let a matrix $A \in \mathbb{R}^{M \times N}$, and $\sigma_i(A)$ be the i-largest singular value of A. Define the operator norm and the trace norm as follow:

$$ \|A\|_2 := \sigma_1(A),\ \|A\|_{tr} = \sum_{i = 1}^{\min\{M, N\}} \sigma_i(A) $$

Could anyone provide me how to prove the below statement?

$$ \langle X,Y\rangle \le \|X\|_{tr}\,\|Y\|_2 $$

Note that $\langle X,Y\rangle $ is the matrix inner product.

Trace norm is equivalent to Frobenius ( http://math.stackexchange.com/questions/725562/equivalence-of-frobenius-norm-and-trace-norm ) and Frobenius is greater than 2-norm ( http://math.stackexchange.com/questions/252819/why-is-frobenius-norm-of-a-matrix-greater-than-or-equal-to-the-2-norm ). Now, all you need is cauchy-schwartz to complete the proof. — TenaliRaman, May 12 '16 at 00:15

Martin Argerami · Answer 1 · 2023-10-04T03:46:36.320

4

This follows from the Singular Value Decomposition. We have $X=VDW$ with $V,W$ unitaries and $D$ "diagonal" (i.e. it is rectangular in general, but the singular values of $X$ lie in the entries with the same row and column number).

I will use the elementary facts that, for any matrix $A$, $|A_{kj}|\leq\|A\|_2$ for all $k,j$; and that the Frobenius norm $\|\cdot\|_2$ is unitarily invariant. So $$ \langle X,Y\rangle=\text{Tr}(Y^TX)=\text{Tr}(Y^TVDW)=\text{Tr}(WY^TV\,D)=\text{Tr}((VYW)^T\,D)\\=\sum_{k=1}^M\sum_{j=1}^N(VYW)^T_{kj}D_{jk}=\sum_{k=1}^{\min{M,N}}(VYW)^T_{kk}\,\sigma_k(X)\leq\|VYW\|_2\,\sum_{k=1}^{\min{M,N}}\sigma_k(X)\\ =\|VYW\|_2\,\|X\|_{tr}=\|Y\|_2\,\|X\|_{\rm tr}. $$

Remark: We can get a tighter (and sharp) inequality if instead we use the estimate $|A_{kj}|\leq\|A\|$, where $\|\cdot\|$ is the operator norm. That way we get $$ |\langle X,Y\rangle|\leq \|Y\|\,\|X\|_{\rm tr}. $$

edited Oct 04 '23 at 03:46

answered Aug 18 '14 at 05:06

Martin Argerami

217,281

What's the difference in the second result? – dmh Oct 04 '23 at 03:09
The operator norm instead of the Frobenius norm. – Martin Argerami Oct 04 '23 at 03:38
$|\cdot|$ is the operator norm? Isnt that the same as what you are calling $|\cdot|_2$? – dmh Oct 04 '23 at 03:40
No. Like I wrote in the answer, $|\cdot|$ denotes the operator norm. It's not the Frobenius norm, which is commonly denoted $|\cdot|_2$. – Martin Argerami Oct 04 '23 at 03:43
But your answer says: "and that the operator norm ‖⋅‖2 is unitarily invariant." Im just confused ( i know it's an old question) – dmh Oct 04 '23 at 03:44
1

Oh, now I see it. That was a typo. My bad. – Martin Argerami Oct 04 '23 at 03:46

Matrix inner product, and operator and trace norm inequality

1 Answers1