# Approximation Error in a Finite Difference Approximation of the Square of Derivative

First Part: (First-order derivative)

Assuming $$f$$ is an infinitely differential function everywhere, the Taylor series of $$f(x + h)$$ at $$x$$ is \begin{align}\tag{1} f(x + h) = f(x) + hf'(x) + \frac{1}{2}h^2f''(\xi) \end{align} where $$\xi$$ is a number between $$x$$ and $$x+h$$.

After rearrangment of terms in (1), we can write $$f'(x) = \frac{f(x+h) - f(x)}{h} - \frac{1}{2}hf''(\xi).$$

Now, we define a finite difference approximation of $$f'(x)$$ by $$f'_h(x) = \frac{f(x+h) - f(x)}{h},$$ and we express $$f'(x) = f'_h(x) + E_1$$ where approximation error $$E_1$$ satisfy \begin{align} |E_1| &= |- 0.5 hf''(\xi)| \\ &\leq Ch \end{align} assuming $$|- 0.5f''(\xi)| \leq C$$. Now, using the definition of Big-O notation, we can say \begin{align}\tag{2} f'(x) = f'_h(x) + O(h) \end{align}

This is a very standard result. However, I have a question for clarification.

Question 1: It seems that the constant $$C$$ can be based on the local behavior of function between $$x$$ and $$x+h$$. Can I say that $$C$$ depends on $$h$$? Moreover, can I comment on the behavior of $$C$$ as $$h \to 0$$?

Second Part: (Square of the first-order derivative)

Using (2), the square of $$f'(x)$$ can be expressed as $$\Big(f'(x)\Big)^2 = \Big(f'_h(x) + O(h)\Big)^2 = \Big(f'_h(x)\Big)^2 + 2f'_h(x)O(h) + O(h^2) = \Big(f'_h(x)\Big)^2 + E_2$$ where the approximation error $$E_2$$ is $$E_2 = 2f'_h(x)O(h) + O(h^2).$$ It seems that, $$E_2$$ depends on the local approximation quantity $$f'_h(x)$$.

Question 2: How can we estimate the leading order term for the error $$E_2$$?

1. It is true that $$C$$ depends on $$x$$ and $$h$$. This implies that if your function has a very poorly behaved second derivative at $$x$$, this method will be inaccurate. The dependance on $$h$$ is also to be expected, since finite differences are based on Taylor series, which converge locally, so our error estimate essentially depends on how well a Taylor series works out to $$x+h$$, which is where that dependency comes from. To your second part of the question, as $$h\to0$$, we have that if $$\xi_h\in[x,x_h]$$ $$\forall h$$, then $$\xi_h\to x$$. If $$u$$ is $$C^2$$, then we can say that $$u''(\xi)\to u''(x)$$ as $$h\to 0$$.
2. For this, this really is the best you can do by simply squaring the forward difference derivative. Try it for yourself on something like $$x^2$$ vs. $$1000x^2$$ at $$x=0$$ vs. $$x=1,10,20$$. At $$x=0$$, $$f'(x)=0$$ so you actually get second order convergence. As you increase $$x$$, your will get first order convergence at each $$x$$, but the prefactor of $$f'(x)$$ gets larger and the results will be worse and worse. $$1000x^2$$ will also do worse at every point because the derivative and second derivative are larger.
• Thanks for your answer. "as $h\to0$, we have that if $\xi_h\in[x,x_h]$ $\forall h$, then $\xi_h\to x$. If $u$ is $C^2$, then we can say that $u''(\xi)\to u''(x)$ as $h\to 0$." - How can we prove this statement? Is this is a standard well-known result? If yes, where can I find it? – hari Apr 4 '19 at 7:43
• It is a combination of 2 results. The first is a property of nested intervals which ensures that $\xi_n\to x$ this is easily proved by contradiction. The second is just putting this limit inside the function $u''$, which we can do if $u''$ is continuous, or equivalently, is $u$ is $C^2$. This can be proven from your favorite definition of continuity and is equivalent to any of them. – whpowell96 Apr 5 '19 at 5:08