First Part: (First-order derivative)

Assuming $f$ is an infinitely differential function everywhere, the Taylor series of $f(x + h)$ at $x$ is \begin{align}\tag{1} f(x + h) = f(x) + hf'(x) + \frac{1}{2}h^2f''(\xi) \end{align} where $\xi$ is a number between $x$ and $x+h$.

After rearrangment of terms in (1), we can write $$ f'(x) = \frac{f(x+h) - f(x)}{h} - \frac{1}{2}hf''(\xi). $$

Now, we define a finite difference approximation of $f'(x)$ by $$ f'_h(x) = \frac{f(x+h) - f(x)}{h}, $$ and we express $$ f'(x) = f'_h(x) + E_1 $$ where approximation error $E_1$ satisfy \begin{align} |E_1| &= |- 0.5 hf''(\xi)| \\ &\leq Ch \end{align} assuming $|- 0.5f''(\xi)| \leq C$. Now, using the definition of Big-O notation, we can say \begin{align}\tag{2} f'(x) = f'_h(x) + O(h) \end{align}

This is a very standard result. However, I have a question for clarification.

Question 1: It seems that the constant $C$ can be based on the local behavior of function between $x$ and $x+h$. Can I say that $C$ depends on $h$? Moreover, can I comment on the behavior of $C$ as $h \to 0$?

Second Part: (Square of the first-order derivative)

Using (2), the square of $f'(x)$ can be expressed as $$ \Big(f'(x)\Big)^2 = \Big(f'_h(x) + O(h)\Big)^2 = \Big(f'_h(x)\Big)^2 + 2f'_h(x)O(h) + O(h^2) = \Big(f'_h(x)\Big)^2 + E_2 $$ where the approximation error $E_2$ is $$ E_2 = 2f'_h(x)O(h) + O(h^2). $$ It seems that, $E_2$ depends on the local approximation quantity $f'_h(x)$.

Question 2: How can we estimate the leading order term for the error $E_2$?


1 Answer 1

  1. It is true that $C$ depends on $x$ and $h$. This implies that if your function has a very poorly behaved second derivative at $x$, this method will be inaccurate. The dependance on $h$ is also to be expected, since finite differences are based on Taylor series, which converge locally, so our error estimate essentially depends on how well a Taylor series works out to $x+h$, which is where that dependency comes from. To your second part of the question, as $h\to0$, we have that if $\xi_h\in[x,x_h]$ $\forall h$, then $\xi_h\to x$. If $u$ is $C^2$, then we can say that $u''(\xi)\to u''(x)$ as $h\to 0$.

  2. For this, this really is the best you can do by simply squaring the forward difference derivative. Try it for yourself on something like $x^2$ vs. $1000x^2$ at $x=0$ vs. $x=1,10,20$. At $x=0$, $f'(x)=0$ so you actually get second order convergence. As you increase $x$, your will get first order convergence at each $x$, but the prefactor of $f'(x)$ gets larger and the results will be worse and worse. $1000x^2$ will also do worse at every point because the derivative and second derivative are larger.

  • $\begingroup$ Thanks for your answer. "as $h\to0$, we have that if $\xi_h\in[x,x_h]$ $\forall h$, then $\xi_h\to x$. If $u$ is $C^2$, then we can say that $u''(\xi)\to u''(x)$ as $h\to 0$." - How can we prove this statement? Is this is a standard well-known result? If yes, where can I find it? $\endgroup$
    – hari
    Apr 4, 2019 at 7:43
  • $\begingroup$ It is a combination of 2 results. The first is a property of nested intervals which ensures that $\xi_n\to x$ this is easily proved by contradiction. The second is just putting this limit inside the function $u''$, which we can do if $u''$ is continuous, or equivalently, is $u$ is $C^2$. This can be proven from your favorite definition of continuity and is equivalent to any of them. $\endgroup$
    – whpowell96
    Apr 5, 2019 at 5:08
  • $\begingroup$ Thanks for the reply. I will look into it more. I have a followup question. The approximation error 𝑂(ℎ) implies that error decays linearly as ℎ→0. Does it mean that I can expect linear behavior only near to zero not otherwise? I mean, I should expect sublinear error performance far from zero, i.e. when ℎ is large. $\endgroup$
    – hari
    Apr 6, 2019 at 6:42
  • $\begingroup$ Quadratic at 0, linear everywhere else $\endgroup$
    – whpowell96
    Apr 6, 2019 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.