# Questions tagged [precision]

Issues related to the representation of numerical quantities in a finite representation in a given base differing from their exact mathematical value.

79 questions
Filter by
Sorted by
Tagged with
915 views

### Higher precision floating-point arithmetic in numerical PDE

I have the impression, from very different resources and talks with researches, that there is a growing demand for high precision computations in numerical partial differential equations. Here, high ...
162 views

### How to include negative number in the log-sum-exp?

I want to know summation of some small numbers, such as {e^-1000, -e^1001, e^1002...} If all numbers are positive, I can use log-sum-exp algorithm. But unfortunately, negative numbers are also ...
154 views

### Compile-time error control vs. interval arithmetic?

I use interval arithmetic for reliable computing. Now, a procedure coded in a good implementation of interval arithmetic takes perhaps about eight times as much as the same procedure carried out ...
67 views

### How error scales with numerical precision in molecular dynamics?

In terms of time-step, numerical error in molecular dynamics scales with square, i.e. $error \approx dt^2$. But how it look for numerical precision ? E.g. how much bigger will be numerical error when ...
1k views

### Why can ill-conditioned linear systems be solved precisely?

According to the answer here, large condition number (for linear system solving) decreases the guaranteed number of correct digits in the floating point solution. Higher order differentiation matrices ...
2k views

### Need for quad precision in scientific computing?

Even if quad precision is not directly supported by most CPUs, many Compilers (GNU, Intel) support them. Also some software packages allow to compile with quad precision, e.g. PETSc. But is there ...
56 views

### Hardware supporting floats with fraction beyond 64 bit

Is there any computation accelerator (like a GPGPU) available, that natively (this means in hardware, not emulated by a library) supports arithmetics using floating point numbers with a fractional ...
102 views

### How can I detect lost of precision due to rounding in both floating point addition and multiplication?

From Computer Systems: a Programmer's Perspective: With single-precision ﬂoating point the expression (3.14+1e10)-1e10 evaluates to 0.0: the value 3.14 is lost ...
3k views

### How can I avoid catastrophic cancellation?

I have the following formula that I need to rewrite in order to avoid catastrophic cancellation. $$y =\sqrt{\frac{1}{2}\left(1-\sqrt{1-x^{2}}\right)}$$ As $x$ becomes smaller, $\sqrt{1-x^{2}}$ ...
5k views

### Is half precision supported by modern architecture?

I am new to computer science and I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is. I thought the 2008 revision of IEEE-754 ...
147 views

### Red flags for numerical computing?

I've learnt the hard way that you should avoid: computing small numbers as the difference of two large numbers evaluating chaotic functions with imprecise inputs. Are there any other red flags a ...
271 views

### Comparison of integrals with a function:

Consider the following integral: $$S(q)=\int_{x=2}^q\sin^2\left(\frac{π\Gamma(x)}{2x}\right)dx$$ And consider the functions : $$R(q)=\frac{q}{\log(q)}$$ $$T(q)=\int_2^q\frac{1}{\log(x)}dx$$ I ...