We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [floating-point]

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

Filter by
Sorted by
Tagged with
20
votes
6answers
3k views

Analyzing Numerical Error in C++ Function

Suppose that I have a function that takes as input several floating-point values (single or double), does some computation, and produces output floating-point values (also single or double). I am ...
13
votes
4answers
6k views

FLOP counting for library functions

When evaluating the number of FLOPs in a simple function, one can often just go down the expression tallying basic arithmetic operators. However, in the case of mathematical statements involving even ...
9
votes
2answers
16k views

How to set double precision values in Fortran

Recently, I've encountered a bizarre problem with FORTRAN95. I initialized variables X and Y as follows: X=1.0 Y=0.1 Later I add them together and print the ...
2
votes
2answers
459 views

Is there a Moore's law for floating-point precision, and what would it imply?

Moore's law states that the number of transistors on an integrated circuit grow exponentially, roughly doubling at a period of 20 months. This affects the amount of memory available and the speed of ...
25
votes
5answers
643 views

Is there software that can autogenerate numerically-accurate floating point C routines from symbolic formulae?

Given a real function of real variables, is there software available that can automatically generate numerically-accurate code to calculate the function over all inputs on a machine equipped with IEEE ...
4
votes
2answers
193 views

Looking for a mathematical proof of stability in floating point arithmetic of CG - any reference?

I am looking for a reference - paper, book, discussion, anything that has a mathematical proof for stability of the conjugate gradient method in floating point arithmetic. Something similar for ...
7
votes
3answers
454 views

Does there exist an arbitrary-precision convex optimization solver?

I have a relatively simple convex optimization problem that involves less than 100 variables but contains a terribly ill-conditioned matrix. I have tried CVX and CPLEX; even though both can typically ...