26
votes
Accepted
How can I avoid catastrophic cancellation?
Take
\begin{align}
1-\sqrt{ 1-x^2} &= (1-\sqrt{ 1-x^2})\frac{1+\sqrt{ 1-x^2}}{1+\sqrt{ 1-x^2}}\\
&= \frac{x^2}{1+\sqrt{ 1-x^2}}
\end{align}
So
\begin{align}
y = x\sqrt{\frac{1}{2+2\sqrt{1-x^2}}...
- 725
15
votes
Practical example of why it is not good to invert a matrix
Here is a quick example which is very practical related to memory usage in PDEs. When one discretizes the Laplace operator, $\Delta u$, for example, in the Heat Equation
$$ u_t = \Delta u + f(t,u) .$$...
- 12.5k
15
votes
Matrix multiplication accuracy Matlab vs Python
First, see Mark L. Stone's answers, which is completely correct. Second, realize that this is the reason why people told you to use relative errors in your numerical analysis class. :)
Third, the ...
- 12.6k
14
votes
Accepted
Whittaker-Shannon interpolation: Accuracy dies with speedup; can it be fixed?
I was able to reproduce the behavior reported in the question, and traced the observed inaccuracies to the following line:
...
- 1,895
14
votes
13
votes
Accepted
Practical example of why it is not good to invert a matrix
Normally there are some principal reasons to prefer solve a linear system respect to use the inverse. Briefly:
problem with the conditional number (@GoHokies comment)
problem in the sparse case (@...
- 1,340
12
votes
Accepted
Accurate and efficient computation of the inverse Langevin function
The inverse Langevin function $\mathcal{L}^{-1}(x)$ is an odd function. Therefore one needs to consider only approximation on the interval $[0, 1]$; the negative half-plane is treated via symmetry ...
- 1,895
12
votes
Accepted
What is the best method of computing $a^{(k)}/k!$?
If the programming environment provides an implementation of the beta function $\mathrm{B}(x,y)$ this computation is straightforward and usually accurate. We have
$$x^{(k)} = \frac{\Gamma(x+k)}{\Gamma(...
- 1,895
9
votes
Accepted
Accurate computation of Gauss-Kuzmin entropy
It's fairly easy to evaluate, to do this expand the logs in Taylor series in $x=(k+1)^{-2}$:
$$ \log_2(1-x) = \frac{-1}{\log 2}\sum_{m\geq1}\frac{x^{m}}{m}$$
$$ \log_2(-\log_2(1-x)) = \frac{\log x}{\...
- 11.5k
9
votes
Accepted
Is it possible to proof a-b+b = a for all double floating-point numbers?
You can sometimes prove such results (or get counterexamples) using an SMT solver such as Z3 that supports floating point arithmetic. Here is a proof of a version of your theorem that says $|((x+y)-y)-...
- 11.5k
8
votes
What is the best method of computing $a^{(k)}/k!$?
Njuffa already answered satisfactorily, but let me comment that dealing with large numbers does not cause loss of precision in floating-point arithmetic: the error is a relative one, corresponding to ...
- 12.6k
8
votes
Unexpected result when summing sorted (and unsorted) positive floating point numbers
Very interesting problem! I might have a partial answer.
To start, I replicated a simple C++ demo that can reproduce the effect
...
- 996
8
votes
Accepted
Does subsampling improve numerical accuracy in Least Squares problems using Normal Equations?
The statement "since I am also concerned about the numerical accuracy of the solution, I was wondering [whether complicated solution is the way to go]" is just a variation of the kinds of ...
- 56.8k
7
votes
Intuition for relative error for vectors
You are overthinking relative error in one-dimension, and I expect that is the source of your confusion.
If I measure the length of an ant, and I am off by 1mm, its a big deal. However, if I were ...
- 171
6
votes
Accepted
General approach to infinite sums
Note the identity for the modified Bessel functions of the first kind,
$
e^z = I_0(z) + 2 \sum_{k=1}^{\infty} I_k(z)
$
(Abramowitz and Stegun, Eq. 9.6.34, https://personal.math.ubc.ca/~cbm/aands/...
- 2,655
5
votes
What is a good definition of "accuracy to N digits"?
Short version
In scientific computing, the notion of relative error is way more popular than accuracy to $N$ digits. Whenever we present the results, we usually plot the obtained (scaled) data and ...
- 8,742
5
votes
Accepted
Does the IEEE-754 standard mandate that exp2 is rounded correctly?
According to [1]: "However, the IEEE-754 standard specifies nothing for elementary functions" and "Indeed, the mathematical libraries (libm) provided by operating systems do not guarantee correct ...
- 2,930
5
votes
Accepted
Adam Bashforth 4 method: how to determine starting values and stil keep the the order of accuracy
What you are looking for is called "bootstrapping". It is a common problem of all multistep ODE integrators and is discussed in many books on the topic.
Among your options are to use a lower-order ...
- 56.8k
4
votes
Automatic finite differences
Potentially related and useful (I gave these resources to my students when teaching Intro to Computational Mathematics, kinda useful pedagogically too):
"Automatic Source-to-Source Error ...
- 2,851
4
votes
Accepted
Automatic finite differences
This is what Griewank et al. call "Piecewise linearization in secant mode", see for instance https://opus4.kobv.de/opus4-zib/files/6164/newton_secant_approx_paper.pdf. The aim of that ...
- 6,159
4
votes
Accepted
Accuracy gap for apparently stable solution
Stability does not necessarily imply accuracy.
I'll demonstrate this with a simple scalar ODE $y' = \lambda y$ (known as Dahlquist's test equation). This simple ODE is generally interpreted as a ...
- 1,973
4
votes
How to get a more accurate cancelation
I am not sure this is possible with the Python libraries since they are using Fortran under the hood and that can't be easily recompiled, but the Julia DifferentialEquations.jl JIT compile specializes ...
- 12.5k
4
votes
Intervals where the sign of a polynomial can be computed reliably
Yes. You can compute a running error bound, i.e, a number $\mu$ such that the difference between the exact value of $y = p(x)$ and the computed value satisfies $\hat{y}$ satisfies $$|y - \hat{y}| \leq ...
- 1,470
3
votes
Accurate evaluation of the sign of a polynomial
Compensated Horner method (http://www-pequan.lip6.fr/~jmc/polycopies/Compensation-horner.pdf) has an error bound of the form
$$ |\mathrm{comphorner}(p, x) - p(x)| \leq u|p(x)| + \gamma_{2n}^2\tilde p(...
- 11.5k
3
votes
Accepted
When writing a research article, how to deal with deviations from theoretical expectations due to numerical errors?
If you have theoretical expectations for some observables of simulations, I see two general ways of dealing with them:
You exploit them to get more accurate results, e.g., you make your algorithm use ...
- 2,147
3
votes
How to find the optimum finite difference method for derivatives?
The choice of finite-difference scheme depends on several factors, such as the smoothness of your data, how uniformly-spaced the data actually is, etc. You may also want to consider just how accurate ...
- 211
2
votes
Accelerating convergence of a generalized continued fraction
The series that converges to $\ln(2)$ appears to be suitable for Cohen-Villegas-Zagier acceleration [PDF]. This is an acceleration technique for alternating series, but continued fractions with ...
2
votes
Accepted
Training accuracy improves but test set accuracy remains the same
I think you experience the implications of overtraining. From this question and this paper:
Since a neural network with a sufficient number of neurons in the hidden layer can exactly implement an ...
- 8,742
2
votes
Which Runge-Kutta method is more accurate: Dormand-Prince or Cash-Karp?
If you are interested in comparing two integrators solve
$$\frac{dq}{dt} = p \\
\frac{dp}{dt} = -q$$
with initial values $(q,p) = (1,0)$ and $dt = 0.1$ for both of them. Then plot the errors
$$...
Only top scored, non community-wiki answers of a minimum length are eligible