How to find the optimum finite difference method for derivatives?

Problem: I have some discrete data of a trajectory $$x_t$$ with errors $$\delta x_t$$ of a physical system sampled at equal times $$\Delta t$$ and I want to calculate the derivative of the trajectory, i.e., the velocity $$v_t$$.

I know I can differentiate it numerically using a finite difference method. From the formulas, it may seem a higher order method is more accurate as its theoretical error is lower. However, I believe the derivative of a function at a given point should depend only on a local region of that point. If we take more points that are far away using a higher order finite difference scheme, I feel we would be increasing the error of the derivative.

Questions: Is this reasoning correct? If so, how to balance these two problems to find the best scheme in a particular situation?

Based on Superbee's answer I have thought of a way to approach the problem:

1. Compare the sampling separation $$\Delta t$$ with the variations on your data $$\Delta x$$. In order to use a higher order method you would need the sampling in time to be much less than the variations in your data. However, to compare the different variables you should remove their units. We could do it in a rough way normalizing against a characteristic length: $$L$$ and $$T$$, for $$x$$ and $$t$$ respectively. These lengths could be defined as an average of the variables. Then, to use higher order methods we would require: $$\frac{\Delta t}{T} \ll \frac{\Delta x}{L.}$$ If we have doubts this is fulfilled, we would prefer a lower order.

2. If we do not consider Runge's phenomenon, in the case 1. is fulfilled we could use higher order methods without a problem. However, there would not be any benefit in using a higher order method if the theoretical error $$\varepsilon$$ of the finite difference method is lower than the error coming from your data $$\delta$$:

• $$\varepsilon$$ depends typically on a derivative of the function (which we could estimate numerically) and a given power of $$\Delta t$$.

• $$\delta$$ comes from the propagation of the error $$\delta x_t$$ to the finite difference method. We could estimate this from the form of the finite difference method using the typical formula of error propagation: $$\delta y = \sqrt{\sum_i \left( \frac{ \partial y }{\partial x_i} \right ) ^2}$$

From these two errors we could estimate also the real error of the derivative.

Does this approach look fine?

• There are many ways to draw a smooth line through existing data points $\{t_i,x_i\}$, but perhaps there are extra constraints that would allow to select more realistic trajectories. For example, some trajectories passing through your data points would have large second derivatives, which would correspond to unphysically large forces that cannot exist in your system. That's an extra constraint that will propagate in the calculation of the first derivatives. Jun 18, 2021 at 13:37

The choice of finite-difference scheme depends on several factors, such as the smoothness of your data, how uniformly-spaced the data actually is, etc. You may also want to consider just how accurate your velocity estimate actually needs to be. For example, if there are large error bars in the trajectory then it probably makes little sense to use a high-order method and then claim that the velocity is also accurate to a high-order.

Other considerations include Runge's phenomenon, the increased computational cost of higher-order methods (due to needing to sample more grid points) and the complications that arise at the boundaries when using a very wide finite-difference stencil.

In summary, I don't think there is a single optimum choice; a lot depends on your data.

Edit in response to comments:

How I would approach this problem may be to not even use finite-differences at all. In my experience finite-differences are used mainly for solving differential equations rather than computing derivatives to discrete experimental data ex post facto. If you are solving differential equations you have the advantage of being able to e.g. solve with different $$h$$ and explicitly check for convergence, but for your problem you don't necessarily have these advantages to verify the "reasonableness" of the finite-difference scheme so it is difficult to say whether any given choice is adequate.

It would certainly be possible to use finite-differences, and the approach in your edits may not be unreasonable, but my instinct for such a problem would be to fit a curve to the data and then just differentiate that (e.g. using polynomial interpolation and then differentiating the polynomial). This may be the simplest way to deal with a host of potential problems with your data: noise (and, therefore, lack of smoothness), steps being unevenly-spaced, how to deal with error bars, different length scales in the data, etc.

• My question is more how would you approach a specific problem. In particular, what would you look to decide which scheme to use. I have edited the question with an idea I have had from your answer. Do you think it looks good? Jun 18, 2021 at 12:41
• @Puco4 See my edited answer in response to your edits/comments Jun 18, 2021 at 16:33
• I have also thought about interpolating a polynomial to the data and then differentiate it. In some articles I have read they remove the noise using a Savitsky-Golay filter with polynomial degree 3. However, if I need to calculate the acceleration I feel differentiating two times a polynomial of degree 3 does not sound too good... Jun 18, 2021 at 18:07
• Again I think the choice of curve fitting method will depend on your data (how much noise etc.) and how much accuracy you really need. Luckily there is quite a lot of work done on curve fitting so I don't think it should be too hard to find a suitable method for your problem. But that constitutes a separate scicomp.SE question entirely Jun 18, 2021 at 23:33
• P.S. it might be interesting to try both numerically differentiation with finite-differences and analytically differentiating a fitted curve, just to see how close they agree and as a sanity check Jun 18, 2021 at 23:35