23

I need to write a program which computes the largest and the smallest (in terms of absolute value) eigenvalues using both power iteration and inverse iteration.

I can find them using the inverse iteration, and I can also find the largest one using the power method. But I have no idea how to find the smallest one using the power method. I tried applying some kind of shift such as $A - \lambda_{max}I$, but to no avail.

How can I modify the power method so that it computes the smallest eigenvalue?

ivt
  • 1,667

4 Answers4

40

If you know that $A$ is symmetric positive-definite, then the spectral shift $B = A-\lambda_\max I$ will work. Use the power method on $B$, then add $\lambda_\max$ to the result to get the smallest eigenvalue of $A$.

The reason this shift works is that a positive-definite matrix has all positive eigenvalues. Therefore $B$ has all non-positive eigenvalues, with the smallest eigenvalue of $A$ now the largest-magnitude (most negative) eigenvalue of $B$. The power method will then find that eigenvalue.

The same approach works for negative-definite matrices, for the same reason.

user7530
  • 50,625
  • This seems to work for symmetric negative-definite matrices as well. – ivt Jan 07 '13 at 01:55
  • Yes. I'll make a note of that. – user7530 Jan 07 '13 at 01:56
  • 1
    While this is the best result we've got so far, any thoughts on symmetric indefinite matrices are still very welcome! – ivt Jan 07 '13 at 02:02
  • 1
    Well, using this approach you can always find the largest (most positive) and smallest (most negative) eigenvalues. But for indefinite matrices it won't help you find the least-magnitude eigenvalue. – user7530 Jan 07 '13 at 02:05
  • 1
    This is true, but it is one thing if you are doing this for fun or homework, but in practice computing the smallest eigenvalue with the power method can be very very slow (I believe it is linear). – csta Apr 30 '14 at 12:46
  • Any chance to get the eigenvector corresponding to the lowest eigenvalue found using that? – kostrykin Oct 23 '15 at 22:26
  • @theV0ID yes, power iteration will also give you the eigenvector – user7530 Nov 16 '16 at 22:03
  • @csta what do you suggest instead? – Him Sep 28 '18 at 03:19
  • I consider a matrix that is symmetric positive definite. The final addition that compensates for the original shift will experience subtractive cancellation if the original matrix has an eigenvalue that is near zero. This significantly reduces the accuracy that can be achieved. If the original matrix has two eigenvalues that are near zero, then convergence will be exceedingly slow for the shifted matrix. Additional details here – Carl Christian Jan 27 '22 at 23:40
11

Suppose $A$ is invertible and has eigenvalue $\lambda$. Then $A^{-1}$ has eigenvalue $\lambda^{-1}$: this follows straight from the eigenvector equation $$Av = \lambda v \Rightarrow \frac{v}{\lambda} = A^{-1}v.$$

Since the smallest eigenvalue of $A$ is the largest eigenvalue of $A^{-1}$, you can find it using power iteration on $A^{-1}$:

$$v_{i+1} = A^{-1} \frac{v_i}{\|v_i\|}.$$

Unfortunately you now have to perform a linear solve at each iteration (or compute a decomposition of $A$), instead of just taking matrix-vector products.

user7530
  • 50,625
  • 1
    Thank you! This is exactly the Inverse Iteration and I already know how to use it. The trick is to find the smallest one using the "pure" Power Method, without inverting $A$ or linear solve at each step. – ivt Jan 07 '13 at 01:21
  • I'm afraid I'm not aware of any "purer" way of getting at the smallest eigenvalue. – user7530 Jan 07 '13 at 01:32
  • I'm reading that using the shift $\sigma = \lambda_{max}$ indeed works, but only for positive-definite matrices. Can someone confirm this? – ivt Jan 07 '13 at 01:39
  • Yes. I'll write up a second answer. – user7530 Jan 07 '13 at 01:41
6

A general creative answer to this question can be composed the following way:

1)Your symmetric/hermitian Matrix $H$ has a spectrum with positive and negative eigenvalues. Assume you can calculate the eigenvalue with maximum absolute value $\omega$ using power method.

2)Shift the matrix by a constant $\lambda$ to target the part of the spectrum that you are interested $H-\lambda I$. In case you are interested in the smallest magnitude eigenvalue $\lambda = 0$.

3)Square your matrix $H'= (H-\lambda I)^2$. This will make your matrix positive definite.

4)Now the desired eigenvalue will be as close to zero, while the change in the largest magnitude eigenvalue can be computed trivially. You can also use the fact that the matrix spectrum will be bounded by the Hilbert-Schmidt norm and avoid step 1.

5)Shift by the maximum eigenvalue/bound $H'=H'- max(spec(H'))I$. Now you will have a negative-definite matrix with the targeted eigenvalue $x'$ having the highest magnitude which you can compute using power-method.

6)Solve for the variable $x$: $x'=(x-\lambda)^2+max(spec(H'))$ You don't have the information of whether your initial eigenvalue was positive or negative but this can be found by multiplying the original matrix to the eigenvector.

In case you have a Hermitian matrix you have to use the inner product defined for complex Hilbert spaces.

Alexis Michailidis
  • 161
3

If your matrix isn't too large, you could do linear regression on $\{I,A,A^2,\cdots,A^n\}$ to find the coefficients of the minimal polynomial. The minimal polynomial will have scalar roots the same as the characteristic polynomial but maybe smaller multiplicities. Thing is this polynomial will solve for the eigenvalues over the scalar field too (Caley-Hamilton theorem links these to each other). Then you can solve numerically over your scalar field for the solution closest to the origin, probably by setting 0 as the first guess to your root-finding method.

This does however deviate a bit from the requirement of using a power iterations for both, so maybe it does not answer your question - but on the other hand it is more general, not assuming any definiteness of the matrix.

mathreadler
  • 26,534