Questions tagged [svd]

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

In linear algebra, the singular-value decomposition (SVD) is a factorization of a real or complex matrix.

Formally, the singular value decomposition of an $m \times n$ real or complex matrix $M$ is a factorisation of the form $UAV^*$ where $U$ is an $m\times m$ real or complex unitary matrix, $A$ is an $m\times n$ rectangular diagonal matrix with non-negative real numbers on the diagonal, and $V$ is an $n\times n$ real or complex unitary matrix.

The singular-value decomposition can be computed using the following observations:

  • The left-singular vectors of $M$ are a set of orthonormal eigenvectors of $MM^*$.
  • The right-singular vectors of $M$ are a set of orthonormal eigenvectors of $M^*M$.
  • The non-zero singular values of $M$ (found on the diagonal entries of $A$) are the square roots of the non-zero eigenvalues of both $M^*M$ and $MM^*$.

Source: Wikipedia.

1566 questions
465
votes
4 answers

What is the intuitive relationship between SVD and PCA?

Singular value decomposition (SVD) and principal component analysis (PCA) are two eigenvalue methods used to reduce a high-dimensional data set into fewer dimensions while retaining important information. Online articles say that these methods are…
217
votes
8 answers

Intuitively, what is the difference between Eigendecomposition and Singular Value Decomposition?

I'm trying to intuitively understand the difference between SVD and eigendecomposition. From my understanding, eigendecomposition seeks to describe a linear transformation as a sequence of three basic operations ($P^{-1}DP$) on a vector: Rotation…
195
votes
7 answers

What is the difference between "singular value" and "eigenvalue"?

I am trying to prove some statements about singular value decomposition, but I am not sure what the difference between singular value and eigenvalue is. Is "singular value" just another name for eigenvalue?
67
votes
4 answers

How unique are $U$ and $V$ in the singular value decomposition $A=U\Sigma V^\dagger$?

According to Wikipedia: A common convention is to list the singular values in descending order. In this case, the diagonal matrix $\Sigma$ is uniquely determined by $M$ (though the matrices $U$ and $V$ are not). My question is, are $U$ and $V$…
64
votes
2 answers

Why does the spectral norm equal the largest singular value?

This may be a trivial question yet I was unable to find an answer: $$\left \| A \right \| _2=\sqrt{\lambda_{\text{max}}(A^{^*}A)}=\sigma_{\text{max}}(A)$$ where the spectral norm $\left \| A \right \| _2$ of a complex matrix $A$ is defined as…
mathemage
  • 850
  • 1
  • 7
  • 15
63
votes
2 answers

What do eigenvalues have to do with pictures?

I am trying to write a program that will perform OCR on a mobile phone, and I recently encountered this article : Can someone explain this to me ?
41
votes
1 answer

Relationship between eigendecomposition and singular value decomposition

Let $A \in \mathbb{R}^{n\times n}$ be a real symmetric matrix. Please help me clear up some confusion about the relationship between the singular value decomposition of $A$ and the eigen-decomposition of $A$. Let $A = U\Sigma V^T$ be the SVD of…
41
votes
5 answers

How can you explain the Singular Value Decomposition to non-specialists?

In two days, I am giving a presentation about a search engine I have been making the past summer. My research involved the use of singular value decompositions, i.e., $A = U \Sigma V^T$. I took a high school course on Linear Algebra last year, but…
36
votes
3 answers

How is the null space related to singular value decomposition?

It is said that a matrix's null space can be derived from QR or SVD. I tried an example: $$A= \begin{bmatrix} 1&3\\ 1&2\\ 1&-1\\ 2&1\\ \end{bmatrix} $$ I'm convinced that QR (more precisely, the last two columns of Q) gives the null space: $$Q=…
whitegreen
  • 1,673
33
votes
7 answers

Understanding the singular value decomposition (SVD)

Please, would someone be so kind and explain what exactly happens when Singular Value Decomposition is applied on a matrix? What are singular values, left singular, and right singular vectors? I know they are matrices of specific form, I know how to…
Celdor
  • 731
  • 1
  • 7
  • 15
33
votes
4 answers

Why are singular values always non-negative?

I have read that the singular values of any matrix $A$ are non-negative (e.g. wikipedia). Is there a reason why? The first possible step to get the SVD of a matrix $A$ is to compute $A^{T}A$. Then the singular values are the square root of the…
Louis
  • 441
32
votes
4 answers

Why does SVD provide the least squares and least norm solution to $ A x = b $?

I am studying the Singular Value Decomposition and its properties. It is widely used in order to solve equations of the form $Ax=b$. I have seen the following: When we have the equation system $Ax=b$, we calculate the SVD of A as $A=U\Sigma V^T$.…
29
votes
4 answers

How does the SVD solve the least squares problem?

How do I prove that the least-squares solution for $$\text{minimize} \quad \|Ax-b\|_2$$ is $A^{+} b$, where $A^{+}$ is the pseudoinverse of $A$?
29
votes
4 answers

Is U=V in the SVD of a symmetric positive semidefinite matrix?

Consider the SVD of matrix $A$: $$A = U \Sigma V^\top$$ If $A$ is a symmetric, positive semidefinite real matrix, is there a guarantee that $U = V$? Second question (out of curiosity): what is the minimum necessary condition for $U = V$?
26
votes
8 answers

Relation between Cholesky and SVD

When we have a symmetric matrix $A = LL^*$, we can obtain L using Cholesky decomposition of $A$ ($L^*$ is $L$ transposed). Can anyone tell me how we can get this same $L$ using SVD or Eigen decomposition? Thank you.
1
2 3
99 100