Approximation Error in a Finite Difference Approximation of the Square of Derivative

Question

First Part: (First-order derivative)

Assuming $f$ is an infinitely differential function everywhere, the Taylor series of $f(x + h)$ at $x$ is \begin{align}\tag{1} f(x + h) = f(x) + hf'(x) + \frac{1}{2}h^2f''(\xi) \end{align} where $\xi$ is a number between $x$ and $x+h$.

After rearrangment of terms in (1), we can write $$ f'(x) = \frac{f(x+h) - f(x)}{h} - \frac{1}{2}hf''(\xi). $$

Now, we define a finite difference approximation of $f'(x)$ by $$ f'_h(x) = \frac{f(x+h) - f(x)}{h}, $$ and we express $$ f'(x) = f'_h(x) + E_1 $$ where approximation error $E_1$ satisfy \begin{align} |E_1| &= |- 0.5 hf''(\xi)| \\ &\leq Ch \end{align} assuming $|- 0.5f''(\xi)| \leq C$. Now, using the definition of Big-O notation, we can say \begin{align}\tag{2} f'(x) = f'_h(x) + O(h) \end{align}

This is a very standard result. However, I have a question for clarification.

Question 1: It seems that the constant $C$ can be based on the local behavior of function between $x$ and $x+h$. Can I say that $C$ depends on $h$? Moreover, can I comment on the behavior of $C$ as $h \to 0$?

Second Part: (Square of the first-order derivative)

Using (2), the square of $f'(x)$ can be expressed as $$ \Big(f'(x)\Big)^2 = \Big(f'_h(x) + O(h)\Big)^2 = \Big(f'_h(x)\Big)^2 + 2f'_h(x)O(h) + O(h^2) = \Big(f'_h(x)\Big)^2 + E_2 $$ where the approximation error $E_2$ is $$ E_2 = 2f'_h(x)O(h) + O(h^2). $$ It seems that, $E_2$ depends on the local approximation quantity $f'_h(x)$.

Question 2: How can we estimate the leading order term for the error $E_2$?

Answer 1

1. It is true that $C$ depends on $x$ and $h$. This implies that if your function has a very poorly behaved second derivative at $x$, this method will be inaccurate. The dependance on $h$ is also to be expected, since finite differences are based on Taylor series, which converge locally, so our error estimate essentially depends on how well a Taylor series works out to $x+h$, which is where that dependency comes from. To your second part of the question, as $h\to0$, we have that if $\xi_h\in[x,x_h]$ $\forall h$, then $\xi_h\to x$. If $u$ is $C^2$, then we can say that $u''(\xi)\to u''(x)$ as $h\to 0$.

2. For this, this really is the best you can do by simply squaring the forward difference derivative. Try it for yourself on something like $x^2$ vs. $1000x^2$ at $x=0$ vs. $x=1,10,20$. At $x=0$, $f'(x)=0$ so you actually get second order convergence. As you increase $x$, your will get first order convergence at each $x$, but the prefactor of $f'(x)$ gets larger and the results will be worse and worse. $1000x^2$ will also do worse at every point because the derivative and second derivative are larger.