1

In Numerical Analysis by Timothy Sauer (Pearson, 2nd Edition) it says that $\tilde{f'}(x+h) = f(x+h) + \epsilon_{\text{mach}}$, where $\tilde{f'}(x)$ is the floating-point representation of the given derivative, $f'(x)$ is the exact value of the derivative and $\epsilon_{\text{mach}}$ is the machine epsilon. I, however, believe that it should say $\tilde{f'}(x+h) = f(x+h) + \frac{\epsilon_{\text{mach}}}{2}$.

To my understanding, the machine epsilon is the difference between any two consecutive floating-point numbers in a given floating-point system. Accordingly, the distance between any real number and the nearest floating point number should, in the worst case, be $\frac{\epsilon_{\text{mach}}}{2}$. For this reason, I fail to see how loss of significance in the floating-point representation of the derivative can cause an absolute error on the size of $\epsilon_{\text{mach}}$. Have I missed something?

Update 1

As has been pointed out in the comments, my understanding of machine epsilon is incorrect and this is the main source of my confusion. If I was to understand the true definition of machine epsilon, the above formula would probably make sense to me.

Furhtermore, an important detail from the book that I did not include is that it says "we will assume that the function errors are on the order of 1 for the present discussion, so that relative and absolute errors are about equal".

  • 2
    $\epsilon_{machine}$ is the distance between 1 and the next largest floating point number. The distance actually changes between floating point numbers since floating point numbers are logarithmicly spaced. https://math.stackexchange.com/questions/1734646/how-are-floating-point-numbers-logarithmically-distributed. In IEEE 754 it is $2^{-52}$ or roughly $10^{-16}$ –  May 21 '19 at 18:57
  • Why do you compare a derivative value with a function value? How long is the way from $x$ to $f(x)$? Usually, each (FPU/elementary) operation contributes a relative error of that size, so that you get a corresponding multiple of $ϵ_{\rm mach}$ as bound for the relative error of the floating point evaluation. Under catastrophic cancellation the relative error might be significantly larger. – Lutz Lehmann May 21 '19 at 19:50

0 Answers0