6

One of the most popular and efficient iterative methods to solve large sparse systems of equations in the least squares sense is LSQR. It is related to CGLS (Conjugate Gradient Least Squares) in that it has the same iterates (mathematically, not numerically).

But what does its name mean? is it

  • Least Squares with QR factorisation? If so, why is it called that, does it provide Q and R? I suppose not because Q would not be sparse.
  • Least SQuaRes?
  • Least SQuares Regression?
  • Something else?

I have looked at the three references over at the SciPy documentation, but none of them seem to explain the naming. In the MATLAB documentation, the call it "the Least Squares Method", even stating it as "... the least squares (LSQR) algorithm...", suggesting that my second option is correct, but with no source.

Jens Renders
  • 4,464
  • 2
    Odd that this is not clear in the literature, but this seems to give an answer; https://www.crewes.org/ForOurSponsors/ResearchReports/2000/2000-18.pdf. This appears to be the source of the algorithm and they do not state their use of the acronym: https://www.stat.uchicago.edu/~lekheng/courses/324/paige-saunders2.pdf – Moo Mar 25 '21 at 12:28
  • @Moo So it seems like the first option is correct, but not because it provides a QR factorisaition of the original system, but because it uses one in each step? – Jens Renders Mar 25 '21 at 12:34
  • @JensRenders: That is what I would go with. It seems to align with what the two papers are saying. – Moo Mar 25 '21 at 12:36

1 Answers1

14

You’re right, the name is not explained anywhere, but there is some logic behind it. When Chris Paige visited Stanford in 1972, we started several methods that needed names. LSQR means that it’s for least-squares problems and uses a QR factorization at each iteration k (updated from the previous iteration).  The QR factorization is used to solve a (k+1) by k least-squares subproblem involving Bk, the lower bidiagonal matrix from the Golub-Kahan bidiagonalization process. This explains the strange name SYMMLQ (for symmetric systems using an LQ factorization of the tridiagonal matrix from the Lanczos process). By this plan, MINRES should have been called SYMMQR, but luckily Chris’s more pronounceable and decipherable name prevailed.

In 1981 I used the unsymmetric orthogonal tridiagonalization for two methods named USYMLQ and USYMQR by the same rule, ultimately described by Saunders, Simon, and Yip in SINUM 1988 (http://dx.doi.org/10.1137/0725052). I couldn’t think of a 6-character name that meant an unsymmetric form of MINRES. (Fortran 66 allowed at most 6 characters for variable and subroutine names.) 

LSQR is equivalent to CG on the normal equations, so it could have been called LSCG.  When the MINRES companion came along (David Fong's PhD thesis), it took many months to think of the name LSMR (SISC 2011 https://doi.org/10.1137/10079687X).

Analogous to LSQR, in SIMAX 2019 there's a paper about LSLQ by Ron Estrin, Dominique Orban, and me (http://www.siam.org/journals/simax/40-1/M111355.html). We have another paper in SIMAX 2019 about LNLQ for the least-norm problem min ||x|| st Ax=b (https://doi.org/10.1137/18M1194948).

It's great to know that you use LSQR every day on large problems. Keep LSMR in mind -- it will always stop sooner because it minimizes ||A'r||, which is most of ||A'r||/(||A|| ||r||),the quantity that both LSQR and LSMR look at to decide when to stop.  The difference can be significant (http://stanford.edu/group/SOL/software/lsmr/).

The same is true for MINRES compared to CG on SPD systems! See Fong and Saunders (2012), "CG versus MINRES: An empirical comparison" (http://stanford.edu/group/SOL/software/minres/).

Best wishes for your use of iterative solvers! Michael

Jens Renders
  • 4,464
Michael Saunders
  • 156