In the Numerical Recipes in section 5.7.- Numerical derivatives it's introduced de roundoff error of:

$$ f^{\prime}(x) \approx \frac{f(x+h)-f(x)}{h} $$

as (with $h$ an "exact" number):

$$ \tag{1} e_{r} \sim \epsilon_{f} \mid f(x) / h| $$

With the fractional accuracy comparable to the machine accuracy $\epsilon_{f} \approx \epsilon_{m}$.

Question: Where does the roundoff error expression $(1)$ come from?

Question: Let's say we have the third degree Taylor expansion:

$$ f(x+h) \approx f(x)+h f^{\prime}(x)+\frac{1}{2} h^{2} f^{\prime \prime}(x)+\frac{1}{6} h^{3} f^{\prime \prime \prime}(x) $$

What is the roundoff error of this Taylor expansion?


1 Answer 1


Let $g(x)=\frac{f(x+h)-f(x)}{h}$, and let $\bar{g}(x)$ its floating point representation with machine precision denoted by $\mu$. Recall that $\text{fl}(f(x)) = f(x)(1+\delta)$ with $|\delta| \leq \mu$.

We have $$|g(x)- \bar{g}(x)| = |\frac{f(x+h)-f(x) - (f(x+h)(1+\delta) - f(x) (1+\delta)}{h})| \leq \frac{2 \mu}{h}$$ where the last inequality follows from $|\delta| \leq \mu$

Notice that the error you'll observe is the following: $|f'(x) - \bar{g}(x)|$ i.e. the difference between the correct result and the representation in finite precision. We can estimate this with a simple trick:

$$|f'(x) - \bar{g}(x)|= |f'(x) - g(x) + g(x) - \bar{g}(x)| \leq C h + \frac{2 \mu}{h}$$ In the last step I used the triangle inequality + the F.D. truncation error (where $C$ of course depends on 2nd derivative of $f$ in the interval ) and the estimate above. In this way you can also compute the optimal $h$, i.e. $\bar{h} = \sqrt{\frac{2 \mu}{C}}$. As a consequence, you'll observe that for $0<h<\bar{h}$ the roundoff error will dominate and you'll keep loosing accuracy

  • $\begingroup$ Thank you. So, could I do something similar with the third degree Taylor expansion? I really don't know how to do it with those derivatives in the expression... $\endgroup$
    – LongJohn
    Apr 25, 2021 at 10:20
  • 1
    $\begingroup$ The roundoff comes from floating point representation. What you can say is that every time you evaluate your function in your machine, you'll obtain $F(x) (1+\delta)$ instead of plain $F(x)$. Of course, every evaluation of a term in the Taylor expansion will not be exact. However, what you're interested in is the impact of the roundoff on the resulting approximation of $f'(x)$, and hence we splitted the two sources of error here: - the FD approximation error (coming from the Taylor expansion, assuming *exact arithmetic") - the error coming from the finite precision of your machine @LongJohn $\endgroup$
    – VoB
    Apr 25, 2021 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.