# Questions tagged [precision]

Issues related to the representation of numerical quantities in a finite representation in a given base differing from their exact mathematical value.

79 questions
Filter by
Sorted by
Tagged with
66 views

### How error scales with numerical precision in molecular dynamics?

In terms of time-step, numerical error in molecular dynamics scales with square, i.e. $error \approx dt^2$. But how it look for numerical precision ? E.g. how much bigger will be numerical error when ...
155 views

### How to include negative number in the log-sum-exp?

I want to know summation of some small numbers, such as {e^-1000, -e^1001, e^1002...} If all numbers are positive, I can use log-sum-exp algorithm. But unfortunately, negative numbers are also ...
55 views

### Hardware supporting floats with fraction beyond 64 bit

Is there any computation accelerator (like a GPGPU) available, that natively (this means in hardware, not emulated by a library) supports arithmetics using floating point numbers with a fractional ...
98 views

### How can I detect lost of precision due to rounding in both floating point addition and multiplication?

From Computer Systems: a Programmer's Perspective: With single-precision ﬂoating point the expression (3.14+1e10)-1e10 evaluates to 0.0: the value 3.14 is lost ...
3k views

### How can I avoid catastrophic cancellation?

I have the following formula that I need to rewrite in order to avoid catastrophic cancellation. $$y =\sqrt{\frac{1}{2}\left(1-\sqrt{1-x^{2}}\right)}$$ As $x$ becomes smaller, $\sqrt{1-x^{2}}$ ...
147 views

### Red flags for numerical computing?

I've learnt the hard way that you should avoid: computing small numbers as the difference of two large numbers evaluating chaotic functions with imprecise inputs. Are there any other red flags a ...
52 views

692 views

### Half precision in Fortran

To improve the time efficiency of my code, I'd like to test a lower precision for real number, using e.g. half precision (2 bytes). However, I'm not sure if I can do that in Fortran. After playing ...
134 views

398 views

### Small, unpredictable results in runs of a deterministic model

I have a sizable model (~5000 lines) written in C. It is a serial program, with no random number generation anywhere. It makes use of the FFTW library for functions using FFT - I do not know the ...