Tag Info

Accepted

How to avoid catastrophic cancellation in python function?

This is indeed called catastrophic cancellation. In fact, this particular case is very easy: rewrite the function using the equivalent, numerically stable expression $$\frac{t}{1+\sqrt{1-t^2}}.$$ ...
• 11.4k
Accepted

When should log1p and expm1 be used?

We all know that $$\exp(x) = \sum_{n=0}^\infty \frac{x^n}{n!} = 1 + x + \frac12 x^2 + \dots$$ implies that for $|x| \ll 1$, we have $\exp(x) \approx 1 + x$. This means ...
• 3,789
Accepted

Is half precision supported by modern architecture?

Intel support for IEEE float16 storage format Intel supports IEEE half as a storage type in processors since Ivy Bridge (2013). Storage type means you can get a memory/cache capacity/bandwidth ...
• 1,991

Meaning of "-0.0" in Python?

Floating point numbers (according to the standard1 nearly all programming languages use) are stored with a certain number of bits in the mantissa, in the exponent, and with a sign bit. As such, ...
• 50.2k
Accepted

Why are log and exp considered 'expensive' computations in ML?

To add to Lutz Lehmann's answer, you can look up the latency for the CPU instructions in this comprehensive table by Agner Fog. For example, on the Intel Ivy Bridge processors: FADD / FSUB (floating ...
• 8,087
Accepted

What are some good strategies to test a floating point arithmetic implementation for double numbers?

You should test transition points. Floating-point numbers have several distinct "ranges": Standard/Normal arithmetic Subnormal arithmetic Infinite arithmetic NaN arithmetic Zero arithmetic ...
• 3,091

Robust computation of the mean of two numbers in floating-point?

I think Higham's Accuracy and Stability of Numerical Algorithms addresses how one can analyze these types of problems. See Chapter 2, especially exercise 2.8. In this answer I'd like to point out ...
• 11.4k
Accepted

Why should I renormalize physical variables?

This question is very closely related to (and possibly a duplicate of) Is variable scaling essential when solving some PDE problems numerically?. There are still good practical reasons to ...
• 29.8k
Accepted

How to determine the amount of FLOPs my computer is capable of

The theoretical peak FLOP/s is given by: $$\text{Number of Cores} * \text{Average frequency} * \text{Operations per cycle}$$ The number of cores is easy. Average frequency should, in theory, factor ...
• 2,981
Accepted

Are BLAS implementations guaranteed to give the exact same result?

No, that is not guaranteed. If you are using a NETLIB BLAS without any optimizations, it it mostly true that the results are the same. But for any practical usage of BLAS and LAPACK one uses a highly ...

Matrix multiplication accuracy Matlab vs Python

Here is R1, as computed in MATLAB: ...
• 1,972

Matrix multiplication accuracy Matlab vs Python

First, see Mark L. Stone's answers, which is completely correct. Second, realize that this is the reason why people told you to use relative errors in your numerical analysis class. :) Third, the ...
• 8,553
Accepted

Hardware performance, floating point functions

In the 1980's era Intel 80x86 architecture, there was a scalar floating point unit that had instructions like FSIN, FCOS, etc. for computing functions like sin and cos. These functions were ...
• 17.6k
Accepted

Evaluating $\log(\exp(x)+1)$ for negative $x$

Use the (IEEE standard) library function log1p, which should be present in all programming languages. The function log1p(x) ...
• 8,553

Why are log and exp considered 'expensive' computations in ML?

$\exp$, $\sin$, $\tan$ and their inverse and otherwise related functions are transcendental, defined by an infinite power series. Meaning it takes some effort to evaluate uniformly good approximations....
• 3,511

Why is $\exp(\ln(x))-x\neq0$ in floating point arithmetic?

The numeric precision is not perfect. You get rounding errors during your computation. When working with floats, don't check if they are = 0, but check if their absolute distance to 0 is smaller than ...
• 121
Accepted

Accuracy loss in single-precision Euclidean norm computation

The squares are harmless (as long as they don't overflow/underflow), because the relative perturbation they introduce is of the order of the machine precision $u\approx 10^{-8}$. Your troubles here ...
• 8,553
Accepted

Accurate Polynomial Evaluation in Floating Point

Horner is indeed the most stable way to evaluate a polynomial (and you get the bonus of evaluating its derivatives with not too much extra cost). Higham presents a nice error analysis of the algorithm ...
• 6,046
Accepted

Small, unpredictable results in runs of a deterministic model

There are aspects of modern computing systems that are inherently non-deterministic that can cause these kinds of differences. As long as the differences are very small in comparison with the ...
• 17.6k