When can I use finite differences for differentiation?

Question

Finite differences are usually used to integrate ODE's and PDE's. However, sometimes they can be used for differentiation which I illustrated simply by using the Matlab code below to differentiate the function $\cos(x)$. I am confused about when I can use finite differences for differentiation, and when I can't. I would like to ask for help to understand, because it may help me to solve a problem I am currently dealing with.

dx = 0.0001;
x  = 0 : dx : 2 * pi;
y  = zeros( 1, length(x) );

for i = 2 : length(x)

    y(i) = ( cos( x(i) ) - cos( x(i - 1) ) ) / dx; 
end

plot(x, y, 'b', 'linewidth', 2);
title('-sin(x) graph');
xlabel('x');
ylabel('y');
grid on

When I was learning how to use finite differences to solve the heat equation, I learned that the exact function $f(x,t)$ can be written as the sum of the numerically approximated function $F(x,t)$ and the error $E(x,t)$:

$$f(x,t) = F(x,t) + E(x,t) \tag 1$$

The stability condition will be satisfied if:

$$ \alpha{\Delta t \over \Delta x^2} < {1 \over 2} \tag 2$$

where $\alpha$ is the heat constant. The interpretation is the following, $\Delta t$ is less than one, so when the error is multiplied by it, it gets smaller. On the other hand, because $\Delta x$ is in the denominator and is also less than one, multiplication of the error with $1/ \Delta x$ causes the error to grow because $1/ \Delta x$ is greater than one. As a rule of thumb, keep the numerator small and the denominator big to achieve numerical stability.

However, in the code that I listed here, there is only $\Delta x$ in the denominator and there is no time step. I thought that doing differentiation this way was not possible because it would cause the error to grow. But that's not the case. I am guessing because the function is analytic and not numerically computed. Am I correct?

I am interested if I can actually solve a PDE using finite difference integration, and then calculate the first spatial derivative of that function using finite differences. Is this possible for the heat and wave equation? If it is, it would be easier for me to solve systems of partial differential equations.

In summary, I would like to know the following things:

1. When can I use finite differences for numerical differentiation? Are there any specific conditions? How exactly does the step size affect the result?
2. Can I use finite differences to calculate the first spatial derivative of a function after I calculated that function using finite difference integration (let's say the function satisfies the heat or wave equation for example)?

Answer 1

1. Whenever the differential is well defined. Note that by definition $$ \lim_{\Delta x \rightarrow 0} \frac{f(x) - f(x-\Delta x)}{\Delta x} = \frac{d f(x)}{dx} $$ If this quantity exists and is finite, then finite difference is "stable" (assuming exact real math), and the first derivative of $f(x)$ is defined as that finite value. As you decrease $\Delta x$ your approximation will become more accurate, and how quickly it converges to the true solution is dependent on the order of accuracy of your finite difference approximation. The above approximation is first order accurate; that is, if we multiply $\Delta x$ by $\frac{1}{2}$ the error will also decrease by a factor of $\frac{1}{2}$ (approximately). You can even go so far as to define "one sided" derivatives and equivalent one sided finite differences.

2. Yes, though keep in mind that doing so will incur an approximation error in addition to whatever errors you've incurred to compute your original solution to the heat/wave equation. Also keep in mind the above condition that the derivative must be defined to begin with.

caveat: Notice that I said assuming exact real math. Floating point numbers approximate mathematics of real numbers, but is not equivalent. For example, if $df/dx = 10^{1000}$ then IEEE-754 double precision floating point numbers cannot physically represent this value. Similarly, if $f(x_0) = 10^{1000}$, trying to evaluate $f(x)$ at $x_0$ will have problems when using double precision even if $df(x_0)/dx$ is something which can be represented using double precision. Unfortunately standard numerical stability theory doesn't really apply here, and you have to rely on floating point truncation error theory to properly analyze whether the finite difference formula will converge or not.

Answer 2

There is a field of numerical analysis called numerical differentiation which studies this kind of problem using various methods, though often things like finite differences. In general, any consistent finite difference scheme (that is, something which in the limit is just the definition of the derivative) will always work for functions which are sufficiently smooth. Numerical stability of finite differences schemes is different, as this says if small errors at some time step grow or decay as you take additional time steps. You can get the order of a finite difference formula through a few different methods, though the standard one is to use Taylor's Theorem and look at how the remainder scales with the step size $\Delta x$.

One caveat is that numerical differentiation often needs many grid points to be accurate, relative to numerical integration. This is the opposite of our analytic/symbolic intuition, for which differentiation is usually easy and integration harder. There are several good sets of online lecture notes which describe these issues in more detail such as these ones.

You may also be interested in VisualPDE which is a website that solves time-dependent PDEs in your web browser via finite differences. You can use an algebraic variable there defined in terms of derivatives of the PDE variables to visualize derivatives of these variables.