Questions tagged [floating-point]

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

Filter by
Sorted by
Tagged with
34
votes
2answers
11k views

When should log1p and expm1 be used?

I have a simple question that is really hard to Google (besides the canonical What Every Computer Scientist Should Know About Floating-Point Arithmetic paper). When should functions such as ...
25
votes
5answers
706 views

Is there software that can autogenerate numerically-accurate floating point C routines from symbolic formulae?

Given a real function of real variables, is there software available that can automatically generate numerically-accurate code to calculate the function over all inputs on a machine equipped with IEEE ...
25
votes
4answers
8k views

How to add large exponential terms reliably without overflow errors?

A very common problem in Markov Chain Monte Carlo involves computing probabilities that are sum of large exponential terms, $ e^{a_1} + e^{a_2} + ... $ where the components of $a$ can range from ...
24
votes
10answers
7k views

Which algorithm is more accurate for computing the sum of a sorted array of numbers?

Given is an increasing finite sequence of positive numbers $z_{1} ,z_{2},.....z_{n}$. Which of the following two algorithms is better for computing the sum of the numbers? ...
21
votes
4answers
72k views

How to determine the amount of FLOPs my computer is capable of

I would like to determine the theoretical number of FLOPs (Floating Point Operations) that my computer can do. Can someone please help me with this. (I would like to compare my computer to some ...
20
votes
6answers
4k views

Analyzing Numerical Error in C++ Function

Suppose that I have a function that takes as input several floating-point values (single or double), does some computation, and produces output floating-point values (also single or double). I am ...
20
votes
4answers
4k views

Is half precision supported by modern architecture?

I am new to computer science and I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is. I thought the 2008 revision of IEEE-754 ...
18
votes
1answer
422 views

Catastrophic cancellation in logsum

I'm trying to implement the following function in double-precision floating point with low relative error: $$\mathrm{logsum}(x,y) = \log(\exp(x) + \exp(y))$$ This is used extensively in statistical ...
17
votes
3answers
2k views

Are BLAS implementations guaranteed to give the exact same result?

Given two different BLAS implementations, can we expect that they make the exact same floating point computations and return the same results? Or can it happen, for instance, that one computes a ...
17
votes
4answers
11k views

Do currently available GPUs support double precision floating point arithmetic?

I have run the molecular dynamics (MD) code GROMACS on a Ubuntu Linux cluster consisting of nodes containing 24 Intel Xeon CPUs. My particular point of interest turns out to be somewhat sensitive to ...
15
votes
7answers
874 views

Robust computation of the mean of two numbers in floating-point?

Let x, y be two floating-point numbers. What's the right way to compute their mean? The naive way ...
13
votes
1answer
4k views

How to avoid catastrophic cancellation in python function?

I am having trouble implementing a function numerically. It suffers from the fact that at large input values the result is a very large number times a very small number. I am not sure if catastrophic ...
13
votes
4answers
536 views

In floating point arithmetic, why does numerical imprecision result from adding a small term to a difference of large terms?

I have been reading the book Computer Simulation of Liquids by Allen and Tildesley. Starting on page 71, the authors discuss the various algorithms that are used to integrate Newton's equations of ...
13
votes
4answers
7k views

FLOP counting for library functions

When evaluating the number of FLOPs in a simple function, one can often just go down the expression tallying basic arithmetic operators. However, in the case of mathematical statements involving even ...
11
votes
2answers
20k views

How to set double precision values in Fortran

Recently, I've encountered a bizarre problem with FORTRAN95. I initialized variables X and Y as follows: X=1.0 Y=0.1 Later I add them together and print the ...
11
votes
1answer
3k views

Meaning of “-0.0” in Python?

We are finding in Python some occasional errors in our coordinate transforms and other similar computations that produce a result of -0.0. What purpose does this ...
11
votes
1answer
277 views

Are there Improved ways of computing $p \log(p)$?

Most math libraries have a number of versions of logarithm functions. Most of the time we assume them to be perfect, but actually quite a lot of them just offer a certain number of digits of precision....
10
votes
3answers
1k views

Relative comparison of floating point numbers

I have a numerical function f(x, y) returning a double floating point number that implements some formula and I want to check that it is correct against analytic ...
10
votes
4answers
705 views

Relevance of fixed-point and arbitrary precision computations

I see very few non-floating point computing libraries/packages around. Given the various inaccuracies of floating point representation, the question arises why there aren't at least some fields where ...
10
votes
1answer
301 views

Order of operations, numerical algorithms

I have read that (1) Ill conditioned operations should be performed before well conditioned ones. As an example, one should calculate $xz-yz$ as $(x-y)z$ since subtraction is ill conditioned while ...
9
votes
4answers
394 views

Small, unpredictable results in runs of a deterministic model

I have a sizable model (~5000 lines) written in C. It is a serial program, with no random number generation anywhere. It makes use of the FFTW library for functions using FFT - I do not know the ...
9
votes
2answers
4k views

Integer operations vs floating point operations

I have been working with an algorithm, which uses additions of floating point vectors, (sparse matrix of floats)x(dense vector of floats) dot products I recently found out that I can get the same ...
9
votes
0answers
664 views

Implementing std::nextafter: Should denormals-are-zero mode affect it? If so, how?

This might be the wrong stackexchange site for this question. math.SE, cs.SE, programmers.SE, and of course stackoverflow are all possibilities. I'm hoping to reach an audience that might actually ...
8
votes
1answer
818 views

Hardware performance, floating point functions

First of all, hope I've found the right forum for this question, if I haven't please pass me on to a one which would fit better. Out of curiosity from an argument with someone who may or may not be ...
8
votes
3answers
977 views

Accurate Polynomial Evaluation in Floating Point

What are the most accurate algorithms for evaluating a polynomial using floating point arithmetic? The internet seems to suggest that Horner's method is commonly used. In particular I have a cubic ...
8
votes
6answers
3k views

Testing equality of two floats: Realistic example

When does it typically make sense in programming to be testing the equality of two floating point numbers? i.e. a == b where both a & b are floats. My ...
8
votes
2answers
844 views

Sum over very small exponentials: Underflow

I am trying to compute (in C) a sum like $S = \sum_i \exp( - a_i )$, where $10^{4} < a_i < 10^{5}$ are approximately normal distributed. So even if I do the Log-Sum-Exp trick $S = \exp(\...
8
votes
1answer
728 views

What's the right way to compare vectors in floating-point?

I know that I should use a tolerance for comparing floating point numbers. But for comparing vectors, I can think of 3 possible solutions corresponding to different distance metrics: Compare the ...
7
votes
5answers
653 views

What algorithm for solving a set of stiff ODEs would be easiest to port to high precision floating point arithmetic?

I want to solve a relatively small system of stiff ODEs (< 10 first-order equations) using high precision floating point arithmetic (using MPFR or alike). What would be the easiest algorithm to ...
7
votes
1answer
285 views

Why should I renormalize physical variables?

I am working with legacy physical codes and I develop new ones based on the output of them. They all use their own internal normalization of variables (for example all distances are divided by the ...
7
votes
2answers
2k views

Choosing epsilons

Most numerical algorithms require an epsilon to be chosen in order to be robust and provide meaningful results. Choosing machine epsilon is usually too aggressive. Barring any special knowledge ...
7
votes
2answers
667 views

transitive floating point comparison with (absolute) tolerance

I want to compare two floating point numbers for equality relative to a known absolute tolerance. However, this is inside an algorithm I wrote quite some time ago, and I believe the logic of that ...
7
votes
1answer
209 views

Where does the floating point error come from? (Finite difference using matrix multiplication versus shifts and adding.)

In Julia it appears that one picks up some error terms when doing finite differences using matrix multiplication versus shifts and addition. ...
7
votes
3answers
487 views

Does there exist an arbitrary-precision convex optimization solver?

I have a relatively simple convex optimization problem that involves less than 100 variables but contains a terribly ill-conditioned matrix. I have tried CVX and CPLEX; even though both can typically ...
7
votes
1answer
151 views

Numerically stable and fast sum of last K elements in sequence

Suppose I have a long, possibly infinite, sequence $x := [x_1, x_2, ...]$, and I want to use it to compute another sequence $y:=[y_1, y_2, ...]$ where each element is the sum of the last K elements of ...
6
votes
2answers
1k views

Matrix multiplication accuracy Matlab vs Python

I am translating some Matlab code into Python and I having some problems regarding matrix multiplication accuracy. Assuming we have following data: ...
6
votes
4answers
767 views

Why is $\exp(\ln(x))-x\neq0$ in floating point arithmetic?

Analytically, the expression $$\exp(\ln(x))-x \enspace,$$ should give 0. However, in Matlab, it does not. x = linspace(1, 10, 10); exp(log(x)) - x; for $x \in ...
6
votes
2answers
291 views

Krylov subspace iterative methods in floating point arithmetic

Is there any work that considers Krylov subspace iterative methods in floating point arithmetic? I'm especially interested in how rounding errors influence the convergence and the accuracy of the ...
6
votes
1answer
1k views

cancellation problem in float-point numbers

In http://en.wikipedia.org/wiki/Floating_point#Addition_and_subtraction, it gives an example about cancellation problem in float-point numbers, see I don't understand why it is written : ...
6
votes
1answer
213 views

Is it possible to proof a-b+b = a for all double floating-point numbers?

I want to know whether the equation : a-b+b = a is always true for a, b belongs to double precision floating-point number and |a|>=|b|. If the equation is true, how can I proof it? If not, what ...
6
votes
2answers
265 views

Is Highams' computation of mean worth the price?

In Accuracy and Stability of Numerical Algorithms, equation 1.6a, Higham gives the following update formula for the mean: $$ M_{1} := x_1, \quad M_{k+1} := M_{k} + \frac{x_k - M_k}{k} $$ Ok, one ...
6
votes
1answer
96 views

Numerical computation of the complex elliptic integral $E(k)$ for medium $|k|$

I have implemented Carlson's algorithm for $E(k)$ from Numerical computation of real or complex elliptic integrals (available from ArXiv eprint, see also DLMF). It is essentially his formula (46) ...
6
votes
1answer
509 views

Using Log Gamma function to avoid overflow

I have to do some numerical calculus using gamma functions. I am using the tgamma incluided in the C++ cmath library. The ...
6
votes
1answer
156 views

Does the IEEE-754 standard mandate that exp2 is rounded correctly?

The IEEE Standard for Floating-Point Arithmetic section "9.2 Recommended correctly rounded functions" lists functions that are recommended (but not required) by a language standard to provide, among ...
6
votes
3answers
149 views

Accurate evaluation of the sign of a polynomial

Let $p$ be a polynomial with floating-point coefficients and let $a$ be a floating-point value. Is there a method for accurately evaluating the sign of $p(a)$ in floating-point arithmetic? I don't ...
6
votes
1answer
207 views

Does mean removal increase accuracy of numerical differentiation?

I wish to compute the derivative of a vector through numerical differentiation. Let's say, we use a standard 2nd order central difference scheme, to arrive at a differentiation matrix, and apply it on ...
6
votes
2answers
2k views

Fast and Numerically Stable Pairwise Distance Algorithms

I'm looking for resources on fast, numerically stable pairwise euclidean distance algorithms. In particular, suppose $A \in \mathbb{R}^{M \times D}$ and $B \in \mathbb{R}^{N \times D}$ are two sets of ...
6
votes
1answer
132 views

Stable computation of ratio of sums of large numbers

I have two sets of large positive numbers $a_1,\ldots,a_n$ and $b_1,\ldots,b_n$. By 'large' I mean of the order of $10^{10}$. I want to calculate the ratio $$R = \frac{a_1 - a_2 + \cdots +(-1)^{n+1}...
6
votes
3answers
2k views

Need for quad precision in scientific computing?

Even if quad precision is not directly supported by most CPUs, many Compilers (GNU, Intel) support them. Also some software packages allow to compile with quad precision, e.g. PETSc. But is there ...
5
votes
3answers
333 views

Machine epsilon does not limit relative rounding error for denormals. Is this a problem?

As we know, machine epsilon limits relative rounding error in the range of normalized floating point numbers. But it is easy to check that this is not true for denormalized numbers. My question is ...