I learned that condition number of an operator/function is a measure of how sensitive the output of the function is with respect to a small change in the input argument.
But then, what is the significance of condition number in the context of numerical methods/analysis? I am not able to see why people emphasize condition number particularly in numerical analysis.
Isn't condition number a characteristic of a given problem regardless of what numerical methods are used to solve the problem?
Condition numbers are relevant because they establish clear boundaries on the accuracy that can be achieved on a given computer architecture.
For the sake of simplicity, let us consider the problem of computing a real function $f : \mathbb{R} \rightarrow \mathbb{R}$. Assuming that $f$ is differentiable, and $x \not = 0$ is such that $f(x) \not = 0$, then the condition number of $f$ is given by $$ \kappa_f(x) = \left|\frac{x f'(x)}{f(x)}\right|.$$ As you said, the condition number measures the sensitivity of the output to small changes in the input. Specifically, if $\bar{x} \approx x$, then $$ \left |\frac{f(x) - f(\bar{x})}{f(x)} \right| \approx \kappa_f(x) \left| \frac{x-\bar{x}}{x}\right| \tag{1}.$$ In popular terms, the condition number magnifies the relative error. If the condition number is large, then we can not afford to be sloppy when computing an approximation $\bar{x}$ of $x$ or $f(\bar{x})$ will be useless as an approximation of $f(x)$.
Now consider an implementation which realizes the function $f$. We seek to compute $y = f(x)$. It is exceedingly unlikely, that our code will return the correct value $y$, rather we will get a good approximation $\hat{y} \approx y$. If our code is backward stable, then $$\hat{y} = f(\hat{x})$$ for a value $\hat{x}$ which is exceedingly close to $x$, ideally, $$ \left| \frac{x-\hat{x}}{x} \right| \leq C u, \tag{2}$$ where $C > 0$ is a modest constant independent of $x$ and $u$ is the unit round off error. In this case, we have $$ \left |\frac{f(x) - f(\hat{x})}{f(x)} \right| \lesssim C \kappa_f(x) u.$$
You are correct that the condition number is a property of the problem independent of the method. But it still gives us important information. A good heuristic is that if the condition number is ~$10^k$, then any numerical method will lose $k$ significant digits in the solution in addition to precision losses due to the algorithm itself (for instance, rounding errors).
To make it more concrete, if you find that the condition number of some problem is ~$10^7$, then you can't use single precision-the solution will lose as many digits as there is precision in the type. That's a nice thing to know before you spend two weeks on a simulation!
