1

I have a question to this post: Show adding rows to a non-singular square matrix will keep or increase its minimum singular value

There Tony asked how to show that the minimum singular value increases or stays the same when adding a row to a non-singular matrix. I'm also interested in how to show this. I understand the answer of loup blanc until he says that the last step is so obvious.

How do you know that $x^T(A^*_1A_1+A^*_2A_2)x\geq x^TA^*_1A_1x$ implies that the $spectrum(A^*_1A_1)\leq spectrum(A^*A)$? As far as I know, you can have a look at the inequality while using an eigenvector of matrix $A_1$ for example. But that doesn't mean that that same vector is also an eigenvector for $A$. So how do you know which one of the singular values is then smaller?

I also tried to describe the eigenvector of the smallest singular value of $A_1$ through a linear combination of the ones eigenvectors of $A$, but that didn't get me anywhere either.

I'd be grateful for any help because I think it shouldn't be that hard to show that this implication hold, but I just can't seem to see how to at the moment.

Kiki
  • 11

1 Answers1

2

An explanation of LB's answer: first of all, his $x^T$ should really be an $x^*$ in this context. Let $\lambda_{\text{min}}(M)$ denote the smallest eigenvalue of a matrix $M$ (LB calls this "min spectrum" instead). Rayleigh's theorem says that if $M$ is Hermitian, we have $$ \lambda_{\min} (M) = \min_{\|x\| = 1} x^*Mx. $$ With that, we can say that $$ \lambda_{\min}(A_1^*A_1) = \min_{\|x\| = 1} x^*(A_1^*A_1)x \leq \min_{\|x\| = 1} x^*(A_1^*A_1 + A_2^*A_2)x = \lambda_{\min}(A_1^*A_1 + A_2^*A_2). $$

Here's an explanation that I find much more intuitive: $$ \begin{align} \sigma_{\min} \pmatrix{A_1\\A_2} &= \min_{\|x\| = 1} \left\| \pmatrix{A_1\\A_2}x\right\| = \min_{\|x\| = 1} \left\| \pmatrix{A_1 x\\A_2 x}\right\| \\&\geq \min_{\|x\| = 1} \left\| \pmatrix{A_1 x\\0}\right\| = \min_{\|x\| = 1} \|A_1x\| = \sigma_\min (A_1). \end{align} $$

Ben Grossmann
  • 234,171
  • 12
  • 184
  • 355
  • sometimes adding an extra step as a bridge can be helpful
    $\min_{|x_1| = 1} x_1^(A_1^A_1)x_1 \leq \min_{|x_1| = 1} x_1^(A_1^A_1)x_1 + \min_{|x_2| = 1} x_2^(A_2^A_2)x_2 \leq \min_{|x_3| = 1} x_3^(A_1^A_1 + A_2^*A_2)x_3$

    because both are positive semidefinite matrices, and 2 choices are better than one

     – user8675309 Feb 19 '20 at 20:33
  • Thank you very much for your response @Omnomnomnom, but I still don't get all of it. I looked up Rayleigh's quotient and found that you can say the following: $\lambda_{min}(M)=v_{\lambda_{min}}^M v_{\lambda_{min}}$. I don't see that you can just exchange $v_{\lambda_{min}}$ through $\underset{||x||=1}{min}x^Mx$.

    I have the same problem with your more intuitive version. How do you know that the first step is valid?

     – Kiki Feb 20 '20 at 08:44
  • Some answers explaining how maximizing/minimizing the Rayleigh quotient gives you a maximal/minimal eigenvalue are given here. Both this characterization of the minimal eigenvalue and the characterization I give of the smallest singular values are standard results; if you'd like a reference, see for instance Horn and Johnson's Matrix Analysis. – Ben Grossmann Feb 20 '20 at 10:41
  • For the more intuitive explanation, is there a situation for $A_2$ such that the $\geq$ becomes $>$? – William Lin Jun 19 '24 at 20:28
  • 1
    @WilliamLin $\geq$ necessarily becomes $>$ when $A_2$ has full column rank (i.e. has linearly independent columns). $A_2$ having full column rank is a sufficient but not a necessary condition. – Ben Grossmann Jun 19 '24 at 20:34
  • Thank you so much, @Ben Grossmann! If I know that $A_1$ and $A_2$ lose 1 rank as a premise, e.g. $A_1, A_2 \in \mathbb{R}^{3 \times 3}$ and I know they are rank 2, does the same statement apply to the second least(the first non-zero) singular value: $\sigma_2 \begin{pmatrix} A_1\ A_2 \end{pmatrix} \geq \sigma_2(A_1)$? If so, does the $\geq$ become $>$ when $A_2$ is exactly rank 2? – William Lin Jun 20 '24 at 13:12
  • For the above case to be true, does it require that $A_1$, $A_2$ and $\begin{pmatrix} A_1\ A_2 \end{pmatrix}$ have the same null space? – William Lin Jun 20 '24 at 17:46
  • 1
    In fact, I think that $\geq$ will become $>$ "most" of the time, whereas the coinciding nullspace requirement is relatively unlikely for random matrices (of the specified rank). I don't know if there are any neat conditions that guarantee strict inequality; I'd suggest that this is worth asking a question about. – Ben Grossmann Jun 20 '24 at 18:28