$2$-norm of a matrix

Question

Given that $\lambda$ is an eigenvalue of $A^T A$ with eigenvector $x$ I need to show

$$\|Ax\|_2^2=\lambda\|x\|_2^2 \tag{1}$$

$$\|A\|_2\le \left\| A^T A \right\|^{1\over 2} \tag{2}$$

I knew that $\|Ax\|_2 = \sum\limits_i\sum\limits_j a_{ij} x_i$.

And what have you tried? Have you written out the explicit meaning of both sides of inequality 1, for example? Have you applied the fact that $\lambda$ is an eigenvalue to try to compare these? — John Hughes, Feb 28 '17 at 12:22
What you say that "know" is wrong, as the example $A = [1]$, $x = [-1]$ shows. — John Hughes, Feb 28 '17 at 12:23
I see, then I must admit that I dont know the definition of this 2-norm of matrix — Myshkin, Feb 28 '17 at 12:24

Andrei Kulunchakov · Accepted Answer · 2017-02-28T12:51:21.257

The first point is proven as follows.

From the SVD of $A = UDV^T$ we can see that eigenvalues of $A^TA = VD^2V^T$ are just squared ones from $A$. At the same time the columns of $V$ are the eigenvectors of $A^TA$. So, exploiting orthogonality of eigenvectors

$$\|Ax\|_2^2 = \|UDVx\|_2^2 = \|D\left(Vx\right)\|_2^2 = \|De_{\lambda}\|x\|\|_2^2 = \|\sqrt{\lambda}\|x\|\|_2^2= \lambda\|x\|^2$$

The proof is based on the property of the second matrix norm

$${\displaystyle \|A\|_{2}={\sqrt {\lambda _{\max }(A^{^{T}}A)}}}$$. $${\displaystyle \|A^{^{T}}A\|_{2}={\sqrt {\lambda _{\max }(A^{^{T}}AA^{^{T}}A)}}}.$$

The same reasoning from the first point says that the eigenvalues of $A^{^{T}}AA^{^{T}}A$ are just the ones of $A^{^{T}}A$ being squared. So $${\displaystyle \|A\|_{2}={\sqrt {\lambda _{\max }(A^{^{T}}A)}}}$$ $${\displaystyle \|A^{^{T}}A\|_{2}={\sqrt {\lambda _{\max }(A^{^{T}}AA^{^{T}}A)}}} = {\sqrt {\lambda _{\max }^2(A^{^{T}}A)}} = \vert\lambda _{\max }(A^{^{T}}A)\vert = \lambda_{\max}(A^TA)=\max(\lambda_{\min}^2(A),\lambda_{\max}^2(A)) \ge \|A\|_{2}^2 $$

Edit

The inequality becomes after the notion that $\lambda_{\max}(A^TA)=\max(\lambda_{\min}^2(A),\lambda_{\max}^2(A))$, because eigenvalues of $A$ could be negative.

Here's a simpler answer for the first one: $|Ax|^2 = | \lambda x |^2 = \lambda^2 \sum x_i^2 = \lambda^2 |x|$, so taking square roots, $|Ax| = |\lambda| |x|$. (Note that the first claim is actually false, by the example I gave in my second comment on the questions, and that your proof implicitly assumes all evalues are nonnegative.) — John Hughes, Feb 28 '17 at 13:06

Sandipan Dey · Answer 2 · 2025-01-16T07:24:00.317

For the problem $(2)$, we have the following:

By definition of 2-norm of a matrix, $\|A\|_2=\underset{\|x\|_2=1}{max}\|Ax\|_2$ , where $x \in \mathbb{R^n}$, $A$ is a $m \times n$ matrix and $Ax \in \mathbb{R^m}$.

Also, by SVD, $A=U\Sigma V^T$, where $U, V$ are $m \times n$ and $n \times n$ unitary (orthonormal) matrices, respectively, and $\Sigma=\begin{bmatrix} \sigma_0 & 0 & \ldots & 0 & \ldots\\ 0 & \sigma_1 & \ldots & 0 & \ldots \\ 0 & 0 & \ldots & 0 & \ldots \\ 0 & 0 & \ldots & \sigma_{n-1} & \ldots \\ \ldots & \ldots & \ldots & \ldots & \ldots \end{bmatrix}$, a $m \times n$ diagonal matrix with $\sigma_0 \geq \sigma_1 \geq \ldots \geq \sigma_n > 0$, with $\sigma_i$ being the singular values of the matrix $A$.

Now, $A^TA=V\Sigma U^TU\Sigma V^T=V\Sigma^2 U^T$, where $U, V$ are unitary with $U^TU=V^TV=I$.

Also, unitary matrices preserves norms, i.e., $\|Ux\|_2=\|x\|_2$ for a unitary matrix $U$.

Hence, By definition of 2-norm,

$\begin{array} \\ \|A^TA\|_2&=&\underset{\|x\|_2=1} {max}\|A^TAx\|_2 \\ &=&\underset{\|x\|_2=1} {max}\|V\Sigma^2U^Tx\|_2 \\ &=&\underset{\|x\|_2=1} {max}\|\Sigma^2U^Tx\|_2 \text{ (since V is unitary, hence norm-preserving)} \\ &=&\underset{\|x\|_2=1} {max}\|\Sigma^2x\|_2 \text{ (since U is unitary)} \\ &=& \sum_{i=0}^{n-1}\sigma_i^2x_i^2 \\ &\leq& \sum_{i=0}^{n-1}\sigma_0^2x_i^2 \text{ (since $\sigma_0 \geq \sigma_i$, $\forall{i}$)}\\ &=& \sigma_0^2\sum_{i=0}^{n-1}x_i^2 \\ &=& \sigma_0^2 \text{ (since $\|x\|_2=1$)} \end{array}$

Also, for a specific $x=e_0=\begin{bmatrix}1\\0\\.\\.\\.\\0\\0\end{bmatrix} \in \mathbb{R}^n$,

$\begin{array} \\ \|A^TA\|_2&=&\underset{\|x\|_2=1} {max}\|A^TAx\|_2 \\ &=&\underset{\|x\|_2=1} {max}\|\Sigma^2x\|_2 \\ &\geq& \|\Sigma^2e_0\|_2 \\ &=& \sigma_0^2 \end{array}$

Hence, combining the above two, $\|A^TA\|_2=\sigma_0^2$

Also, from here, we have, $\|A\|_2=\sigma_0$, the largest singular value.

$2$-norm of a matrix

2 Answers2