We’re rewarding the question askers & reputations are being recalculated! Read more.

# Questions tagged [floating-point]

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

107 questions
Filter by
Sorted by
Tagged with
44 views

### Matrix multiplication not working in Scilab

I entered an instruction to calculate the coordinates of a vector after a change of basis in order to repeat it many times with various vectors. X0=[1;1/2] is a ...
62 views

### How can I calculate the exponential integral?

(I originally asked this in a different exchange.) I'm writing a program that uses the prime-counting function. Right now, I'm using x/log(x), but I want to ...
404 views

2k views

### Testing equality of two floats: Realistic example

When does it typically make sense in programming to be testing the equality of two floating point numbers? i.e. a == b where both a & b are floats. My ...
90 views

### Numerical evaluation of gaussian-like integral expressible as a recurrence relation

I'm looking to numerically evaluate $\log f_p(z)$ and its derivative $f^\prime_p(z)/f_p(z)$ accurately and efficiently in floating-point, where  f_p(z)=\int_0^\infty r^{p-1} \exp\left(-\tfrac{1}{2} ...
88 views

### Do BLAS routines compute their respective operations with minimum error?

Do all BLAS routines compute the respective operation with minimum error ? i.e. Is the reduction in sdot computed with least error ? I need to call these ...
347 views

### Can floating point error (in FFTW3) cause non-deterministic behavior?

I am solving a numerical optimization problem with my own L-BFGS (implemented in c++). The problem has $\approx 10^6$ optimization parameters. To find the objective function gradient, I am taking a ...
310 views

### Small, unpredictable results in runs of a deterministic model

I have a sizable model (~5000 lines) written in C. It is a serial program, with no random number generation anywhere. It makes use of the FFTW library for functions using FFT - I do not know the ...
I have simple code, which flags nodes with in region enclosed by cylinder. On implementing the code, the result is mild tilt of the cylinder observed case with $\theta=90^{\circ}$. The algorithm for ...