4
$\begingroup$

I try to solve a system of coupled equations, where a very nasty division operation occurs. In fact, I need to compute a derivative of two exponential decaying functions. Let's illustrate this with the following example which can easily computed in an analytical way. Assume we have given the function $$f(z) = e^{-2z^2}$$ and compute the quotient $$ \frac{\partial_z f(z)}{f(z)} = \frac{-4 z e^{-2z^2}}{e^{-2z^2}} = -4 z . $$ I'm considering a symmetric interval around $0$ and I only have given the function $f$, so I need to compute its derivate $\partial_z f$ (so far I use spectral methods to do this, but I also tried other methods without any significant difference). At some magnitude of $|z|$ the rounding errors during the computation of the derivate $\partial_z f(z)$ become comparable to the function values of $f(z)$ and the quotient does not longer represent any meaningful.Analytical Quotient - Numerical Quotient

In order to get rid of the "error noise" in my numerical solution I tried to work with low pass filters. Since I know which characteristic the solution should have I divided the singal in to parts, the left(this one reversed) and rigth one and it tunred out it work not as expected. The higher oscillatory frequencies are damped pretty well according to the order of the filter, but unforunatly the filtered signal is averaged around zero on the ouside and in the middle (where the sperated left and right part are glued togehter) the function has a big plateu where it is zero. Which in total causes the approximation is not even close to adequate. enter image description here

Currently i don't see any other option to perfrom this computation numericaly, why I decided to ask this question here. Maybe there is a standart ansatz to solve problems like this, which I don't have heared from so far. I would be very thankful for your input what I could try else.

Cheers

Improve this question
$\endgroup$
4
  • $\begingroup$ I am not sure I get this right, but if you only need the value of this derivative on a set of discrete points, you can try finite differences (which may not work if your function is really not "well behaved"), or complex step, which is an extension of the finite difference method, but with a complex perturbation instead, allowing to reach machine accuracy for the pointwise estimation of the derivative of a real function. If that suits your need, I can link you to some references. You can also try automatic differentiation maybe. $\endgroup$
    – Laurent90
    Commented Nov 19, 2020 at 19:35
  • $\begingroup$ I almost achieve machine precision, the problem is that at some magnitude the function value gets close to machine precision and that's the point where the problems start (see in the first figure, where the high oscillations start) since the error is now of the same magnitude as the function value. $\endgroup$
    – Hamilcar
    Commented Nov 19, 2020 at 21:03
  • $\begingroup$ Maybe I still did not get it, but I've tried your example with finite differences or complex step, and it works like a charm... What's the method that gave the erratic result from your post ? $\endgroup$
    – Laurent90
    Commented Nov 19, 2020 at 22:15
  • $\begingroup$ Actually troubles appear when $|z|>15$, which is already better than with the method you showcased. Then the value of $f$ is 0 up to machine precision therefore the computation yields 0/0=NaN. I think the only way to overcome this is to use analytical formulae, or maybe automatic differentiation can do the trick as well. $\endgroup$
    – Laurent90
    Commented Nov 19, 2020 at 22:29

2 Answers 2

Reset to default
3
$\begingroup$

Just take the log of your function, and then take the derivative of it.

If the actual function is indeed positive definite as in the example then it is identically the same as the quotient that we need to calculate but without division,

$\partial_z \ln(𝑓)= \frac{ 1} {f} \partial_z 𝑓$

If the actual function is not positive-definite then it may become a bit more complicated as you'd have to use the absolute value and track the sign separately.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ Though that works in theory, numerically we are still confronted with the problem that $f$ goes to 0 so quickly that taking is logarithm and differentiating it numerically becomes nonsense from $|z| \approx 15$ onwards. $\endgroup$
    – Laurent90
    Commented Nov 20, 2020 at 7:56
  • 1
    $\begingroup$ @Laurent90 Can you show an example of a function that you have in mind? For the kind of function as in the provided example $f \sim \exp(-z^2)$ it would not be numerical nonsense if you take the log of it; it would be just a nice quadratic function that can be differentiated numerically as much as you like. $\endgroup$
    – Maxim Umansky
    Commented Nov 20, 2020 at 16:05
  • 1
    $\begingroup$ Yes, provided you can derive "by hand" the expresion of $log(f)$. If the actual $log(f)$ the OP uses can be derived analytically, then the quotient $\frac{\partial_z f}{f}$ could most likely also be derived analytically. However, if the function is too complex such that this operation is not possible, then evaluating $log(f(z))$ first requires evaluating $f(z)$, and therefore numerical problems when $z$ is so high that $f(z)=0$ to machine precision... Then the only solution I can think of is to use higher precision numbers, or maybe automatic differentiation... What's your thought ? $\endgroup$
    – Laurent90
    Commented Nov 20, 2020 at 17:15
  • 3
    $\begingroup$ It would be helpful to see what the actual f(z) looks like. $\endgroup$
    – Maxim Umansky
    Commented Nov 20, 2020 at 19:06
2
$\begingroup$

We do not know what your equations look like exactly, but maybe you may get away with transforming your whole coordinate system to more meaningful units. I mean, if your units are for example au (distance earth sun), and you simulate something very small, you run into these floating point problems earlier than if you'd transform your problem to (mili/micro/nano/...)meters. This will not actually solve the underlying problem, but may push the errors you observe into a range that is not relevant anymore (see: https://stackoverflow.com/questions/7006510/density-of-floating-point-number-magnitude-of-the-number). This is also one of the less obvious advantages of simulating in nondimensionalized coordinates, because here your numerical values will be closer to unity.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.