# Taylor expansion round-off error

In the Numerical Recipes in section 5.7.- Numerical derivatives it's introduced de roundoff error of:

$$f^{\prime}(x) \approx \frac{f(x+h)-f(x)}{h}$$

as (with $$h$$ an "exact" number):

$$\tag{1} e_{r} \sim \epsilon_{f} \mid f(x) / h|$$

With the fractional accuracy comparable to the machine accuracy $$\epsilon_{f} \approx \epsilon_{m}$$.

Question: Where does the roundoff error expression $$(1)$$ come from?

Question: Let's say we have the third degree Taylor expansion:

$$f(x+h) \approx f(x)+h f^{\prime}(x)+\frac{1}{2} h^{2} f^{\prime \prime}(x)+\frac{1}{6} h^{3} f^{\prime \prime \prime}(x)$$

What is the roundoff error of this Taylor expansion?

Let $$g(x)=\frac{f(x+h)-f(x)}{h}$$, and let $$\bar{g}(x)$$ its floating point representation with machine precision denoted by $$\mu$$. Recall that $$\text{fl}(f(x)) = f(x)(1+\delta)$$ with $$|\delta| \leq \mu$$.

We have $$|g(x)- \bar{g}(x)| = |\frac{f(x+h)-f(x) - (f(x+h)(1+\delta) - f(x) (1+\delta)}{h})| \leq \frac{2 \mu}{h}$$ where the last inequality follows from $$|\delta| \leq \mu$$

Notice that the error you'll observe is the following: $$|f'(x) - \bar{g}(x)|$$ i.e. the difference between the correct result and the representation in finite precision. We can estimate this with a simple trick:

$$|f'(x) - \bar{g}(x)|= |f'(x) - g(x) + g(x) - \bar{g}(x)| \leq C h + \frac{2 \mu}{h}$$ In the last step I used the triangle inequality + the F.D. truncation error (where $$C$$ of course depends on 2nd derivative of $$f$$ in the interval ) and the estimate above. In this way you can also compute the optimal $$h$$, i.e. $$\bar{h} = \sqrt{\frac{2 \mu}{C}}$$. As a consequence, you'll observe that for $$0 the roundoff error will dominate and you'll keep loosing accuracy

• Thank you. So, could I do something similar with the third degree Taylor expansion? I really don't know how to do it with those derivatives in the expression... Apr 25 at 10:20
• The roundoff comes from floating point representation. What you can say is that every time you evaluate your function in your machine, you'll obtain $F(x) (1+\delta)$ instead of plain $F(x)$. Of course, every evaluation of a term in the Taylor expansion will not be exact. However, what you're interested in is the impact of the roundoff on the resulting approximation of $f'(x)$, and hence we splitted the two sources of error here: - the FD approximation error (coming from the Taylor expansion, assuming *exact arithmetic") - the error coming from the finite precision of your machine @LongJohn
– VoB
Apr 25 at 10:28