A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

learn more… | top users | synonyms

3
votes
0answers
31 views

Numerical computation of the complex elliptic integral $E(k)$ for medium $|k|$

I have implemented Carlson's algorithm for $E(k)$ from Numerical computation of real or complex elliptic integrals (available from ArXiv eprint, see also DLMF). It is essentially his formula (46) ...
3
votes
1answer
84 views

Stabilizing a 3x3 real symmetric matrix eigenvalue calculation

I have many 3x3 real symmetric matrices for which I need to determine the eigenvalues. Wikipedia gives a nice non-iterative algorithm for this case, which I have translated into C++: ...
2
votes
3answers
109 views

Accurate computation of the current time in time integrator

I implement Runge--Kutta method for time integration of the system of nonlinear conservation laws $$ u_t + f(u)_x = 0. $$ As the system is nonlinear, we have to recompute time step ...
0
votes
1answer
36 views

Why the greatest exponent that can be represented in single-precision floating-pointer numbers is 127 (and not 128)?

I was told by a class mate that the smallest exponent that we can represent by a single-precision floating-point number (which uses 8 bits for the exponent) is $-126$ and the greatest is $127$. I ...
0
votes
0answers
16 views

t digits are used to represent the mantissa in floating-point system, but rounding unit is calculated for doubles with 53 bits

I'm reading the book "A First Course in Numerical Methods" by U. Ascher and C. Greif, and in the 2nd chapter it's written that ... we associate $x$ a floating point representation $fl(x)$ of the ...
2
votes
1answer
47 views

Is more robust to digit cancellation the sum operator or the mean operator?

Note: this question is not strictly related to Matlab or any other environment (even though I would prefer that you refer to my Matlab code). I've n = 10000 ...
1
vote
1answer
39 views

How to compute $\mathrm{proj}_{SDP}(C\odot X)./C$ without numerical problems?

I have a matrix, $X$, it is symmetric. I project $C \odot X$ and $D\odot X$ to semidefinite cone. $C$ is a Gramian matrix with some elements near zero and of course semidefinite, with one row and ...
1
vote
1answer
78 views

Addition and subtraction of two floats in Python

Yesterday I was wondering how floats are handled in a computer and what they look like in binary... I learnt about the single-precision floating-point and I tried to see the limit of that format... I ...
3
votes
0answers
31 views

Implementing std::nextafter: Should denormals-are-zero mode affect it? If so, how?

This might be the wrong stackexchange site for this question. math.SE, cs.SE, programmers.SE, and of course stackoverflow are all possibilities. I'm hoping to reach an audience that might actually ...
7
votes
1answer
311 views

Hardware performance, floating point functions

First of all, hope I've found the right forum for this question, if I haven't please pass me on to a one which would fit better. Out of curiosity from an argument with someone who may or may not be ...
6
votes
2answers
85 views

Sum over very small exponentials: Underflow

I am trying to compute (in C) a sum like $S = \sum_i \exp( - a_i )$, where $10^{4} < a_i < 10^{5}$ are approximately normal distributed. So even if I do the Log-Sum-Exp trick $S = ...
2
votes
1answer
47 views

Comparing two versions of the same hydrodynamic code and their error

So I have two versions of a hydrodynamic code that has the same underlying physics. Lets call them code A and B. However code B is more optimized and more object oriented. I was trying to compare the ...
1
vote
1answer
57 views

How do I add some floating point numbers, keeping numerical accuracy in mind?

I am solving a problem involving the line with the set of points $(x_3,y_3)$ that are equidistant to two given points $(x_1,y_1)$ and $(x_2,y_2)$. The equation for this line is $$(x_3 - x_1)^2 + (y_3 ...
2
votes
1answer
55 views

overflow upper incomplete gamma function

I want to calculate the following equation: $$\frac{\theta \Gamma \left(k+1,\frac{o}{\theta }\right)-o \Gamma \left(k,\frac{o}{\theta }\right)}{\Gamma (k)}+o+s$$ with $s>0, o>0, k>0, ...
3
votes
1answer
56 views

IEEE-754 NaNs and missing data

I would like -if possible at all- to represent and handle missing data (in the statistical sense) within the standard IEEE-754 format. Seeing that for both SNaNs and QNaNs various bit representations ...
2
votes
2answers
249 views

what does -ffast-math do?

What kind of optimisations does the option -ffast-math do ? I saw that the time taken for a simple $O(n^2)$ algorithm being reduced to that of an $O(n)$ algorithm ...
3
votes
1answer
54 views

Do BLAS routines compute their respective operations with minimum error?

Do all BLAS routines compute the respective operation with minimum error ? i.e. Is the reduction in sdot computed with least error ? I need to call these ...
1
vote
2answers
115 views

performance of icc main.cpp == g++ -ffast-math main.cpp

I have a program that has a nested loop, together with its parent running at $O(n^2)$ complexity performing floating point arithmetic. I see that the performance of the code when compiled with ...
6
votes
1answer
100 views

Using Log Gamma function to avoid overflow

I have to do some numerical calculus using gamma functions. I am using the tgamma incluided in the C++ cmath library. The ...
11
votes
1answer
240 views

When should log1p and expm1 be used?

I have a simple question that is really hard to Google (besides the canonical What Every Computer Scientist Should Know About Floating-Point Arithmetic paper). When should functions such as ...
12
votes
6answers
673 views

Robust computation of the mean of two numbers in floating-point?

Let x, y be two floating-point numbers. What's the right way to compute their mean? The naive way ...
3
votes
0answers
105 views

Rounding errors in images of Julia sets

One typically computes Julia sets by iterating a complex function, such as a polynomial or rational function. How do rounding errors affect the results? I'm looking for references on this issue, ...
2
votes
2answers
143 views

FLOPs of iterative vs direct solvers

In general, do iterative solvers require more floating point operations than the direct solver counterparts? I have some scientific code (written in both PETSc and FEniCS) for solving a mixed FE ...
1
vote
3answers
108 views

Matlab Large Numbers and Small Numbers

I recently have been assigned a project to calculate the roots of a cubic polynomial. However, the issue is that the roots could be very big, but also extremely small. I've been trying to use ...
5
votes
2answers
102 views

Computation of multipole expansion of potential not converging

According to Beatson and Greengard's short course on FMM: ( Eq. 5.15 & 5.16 setting k=1, q=1 ) We can approximate a potential $\phi = 1/(r-R)$ using: $$ {1\over |\vec{r}-\vec{R}|} = ...
2
votes
1answer
93 views

What are the tradeoffs of using statically allocated arrays vs. pointers and dynamic allocation? [closed]

I am learning Monte Carlo simulation by C++. I begin with reading codes (from the internet and text books) of the 2D Ising model and the XY model. I find some people define spins simply by a two ...
5
votes
1answer
100 views

Additional cost associated with quad (or higher) precision

In going from double precision to quad (or higher) precision, roughly how much of a performance hit is taken on various architectures and frameworks. Do floating point operations take twice as much ...
2
votes
1answer
138 views

On the cardinality of the set of numbers representable by a floating point system

I am new to numerical methods and I cannot think of a way to start solving the following problem. I know the mantissa, base and the boundaries are related when defining a number but I cannot really ...
4
votes
2answers
99 views

Difference between rounding modes in computational science?

Are there any instances of scientific numerical problems where the choice of rounding mode matters? There are usually a number of different rounding modes available: to $0$, away from $0$, to ...
7
votes
1answer
201 views

Why should I renormalize physical variables?

I am working with legacy physical codes and I develop new ones based on the output of them. They all use their own internal normalization of variables (for example all distances are divided by the ...
11
votes
1answer
1k views

How to avoid catastrophic cancellation in python function?

I am having trouble implementing a function numerically. It suffers from the fact that at large input values the result is a very large number times a very small number. I am not sure if catastrophic ...
0
votes
0answers
378 views

What is the smallest positive integer that is not exactly representable as a floating point number in this system?

I have difficulties answering Exercise 3.10 from the book Overton M. Numerical computing with IEEE floating point arithmetic, 2nd edition. What is the smallest positive integer that is not exactly ...
0
votes
0answers
21 views

Compute hypergeometric function ratio: $\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$, for large positive $a,b,c$? [duplicate]

I need a numerically stable way to compute the following ratio: $$\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$$ All the parameters are real positive numbers, with $0 < a,b,c$ and $0 < x ...
2
votes
2answers
139 views

Choosing epsilons

Most numerical algorithms require an epsilon to be chosen in order to be robust and provide meaningful results. Choosing machine epsilon is usually too aggressive. Barring any special knowledge ...
3
votes
1answer
134 views

Compute hypergeometric function ratio: $\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$?

I need a numerically stable way to compute the following ratio: $$\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$$ All the parameters are real numbers, with $a< 0$,$\ $ $b,c > 0$ and ...
3
votes
0answers
44 views

What disables Gradual Overflow?

After studying the process of Gradual Underflow, I'm left a little curious as to why machines don't implement Gradual Overflow; where numbers exceeding the overflow level would be stored as ...
0
votes
1answer
295 views

Overcoming floating point issues when Inverse does not exist but determinant provides nonzero result

I have an expression (let's say determinant of matrix A) expressed in symbolic form in terms of 2 decision variables x1, x2 and 2 parameters q1 and q2. I'm minimizing this using fmincon for different ...
5
votes
1answer
133 views

Accurate Polynomial Evaluation in Floating Point

What are the most accurate algorithms for evaluating a polynomial using floating point arithmetic? The internet seems to suggest that Horner's method is commonly used. In particular I have a cubic ...
7
votes
3answers
16k views

How to determine the amount of FLOPs my computer is capable of

I would like to determine the theoretical number of FLOPs (Floating Point Operations) that my computer can do. Can someone please help me with this. (I would like to compare my computer to some ...
16
votes
9answers
3k views

Which algorithm is more accurate for computing the sum of a sorted array of numbers?

Given is an increasing finite sequence of positive numbers $z_{1} ,z_{2},.....z_{n}$. Which of the following two algorithms is better for computing the sum of the numbers? ...
2
votes
2answers
86 views

Re-scaling array of floats so that all items are approximately integer

I have an array of floating point values $F$. I want to input my array into an algorithm that only takes integer values. How can I efficiently determine the smallest multiplier $m$ such that all ...
5
votes
3answers
177 views

Machine epsilon does not limit relative rounding error for denormals. Is this a problem?

As we know, machine epsilon limits relative rounding error in the range of normalized floating point numbers. But it is easy to check that this is not true for denormalized numbers. My question is ...
2
votes
1answer
1k views

range of positive mantissa in given floating-point number representation

I am a student and I came to this question while solving problems regarding the float-points. ...
4
votes
1answer
4k views

Machine epsilon (eps)

The wiki for machine epsilon says: "Machine epsilon gives an upper bound on the relative error due to rounding in floating point arithmetic" If machine epsilon is the upper bound on the relative ...
1
vote
2answers
696 views

How to handle floating point operations in HLSL?

I'm trying to write a perona malik anisotropic diffusion filter for the GPU. I'm basing my shader off a matlab implementation of the filter. I'm running into trouble because of what I suspect is ...
16
votes
4answers
7k views

Do currently available GPUs support double precision floating point arithmetic?

I have run the molecular dynamics (MD) code GROMACS on a Ubuntu Linux cluster consisting of nodes containing 24 Intel Xeon CPUs. My particular point of interest turns out to be somewhat sensitive to ...
9
votes
4answers
428 views

Relevance of fixed-point and arbitrary precision computations

I see very few non-floating point computing libraries/packages around. Given the various inaccuracies of floating point representation, the question arises why there aren't at least some fields where ...
6
votes
1answer
410 views

cancellation problem in float-point numbers

In http://en.wikipedia.org/wiki/Floating_point#Addition_and_subtraction, it gives an example about cancellation problem in float-point numbers, see I don't understand why it is written : ...
11
votes
1answer
217 views

Are there Improved ways of computing $p \log(p)$?

Most math libraries have a number of versions of logarithm functions. Most of the time we assume them to be perfect, but actually quite a lot of them just offer a certain number of digits of ...
5
votes
2answers
947 views

Need for quad precision in scientific computing?

Even if quad precision is not directly supported by most CPUs, many Compilers (GNU, Intel) support them. Also some software packages allow to compile with quad precision, e.g. PETSc. But is there ...