Prove that $\operatorname{cond}(f ) = \|A^{−1}\|\|A\|.$

Question

Question:

I am trying to prove that the condition number of a function $ f(x) = Ax $, where A $\in R^{m\times n}$ and $x \in R^{n\times 1}$ , is given by:

$$ \text{cond}(f) = \|A^{-1}\|\|A\| $$

The given definition of the condition number of a function ( f ) is:

$$ \limsup_{\|\Delta x\| \to 0} \frac{\frac{\|\Delta y\|}{\|y\|}}{\frac{\|\Delta x\|}{\|x\|}} $$

Starting from this definition, I have:

$$ \text{cond}(f) = \limsup_{\|\Delta x\| \to 0} \frac{\frac{\|A\Delta x\|}{\|Ax\|}}{\frac{\|\Delta x\|}{\|x\|}} $$

Then, I proceed with the following simplifications:

$$ \limsup_{\|\Delta x\| \to 0} \frac{\|A\|}{\|A\|} \frac{\|\Delta x\|}{\|x\|}\frac{\|x\|}{\|\Delta x\|} $$

Next, eliminating ( x ), I arrive at:

$$ \limsup_{\|\Delta x\| \to 0} \frac{\|A\|}{\|A\|} $$

Finally, I get:

$$ \text{cond}(f) = \|A^{-1}\|\|A\| $$ However, I feel like something is wrong, but I’m not sure what the exact issue is. Could anyone help me identify where my reasoning or steps might be incorrect?

Answer 1

The condition number (at least one of the myriad definitions) determines the relative sensitivity of solutions to $Ay=x$, so you presumably need $f(x) = A^{-1} x$.

One issue with your definition is that $x$ is fixed, so the best we can get is $\limsup_{h \to 0} { { \|A^{-1} h \|\over \|A^{-1}x\| } \over { \|h\| \over \|x\|} } = \limsup_{h \to 0} { \|A^{-1} h \| \over \|h\| } { \|x\| \over \| A^{-1} x\| } = \|A^{-1} \| { \|x\| \over \| A^{-1} x\| } $.

If you allow $x$ to vary , we get $\sup_{x \neq 0} \|A^{-1} \| { \|x\| \over \| A^{-1} x\| } = \sup_{w \neq 0} \|A^{-1} \| { \|Aw\| \over \| w\| } = \|A^{-1}\| \|A\|$.

The term condition number is defined in many different ways, for example, Higham.