# Questions tagged [precision]

Issues related to the representation of numerical quantities in a finite representation in a given base differing from their exact mathematical value.

79 questions
Filter by
Sorted by
Tagged with
5k views

### How can the gravitational n-body problem be solved in parallel?

How can the gravitational n-body problem be solved numerically in parallel? Is precision-complexity tradeoff possible? How does precision influence the quality of the model?
4k views

### What's the state-of-the-art in highly oscillatory integral computation?

What's the state-of-the-art in the approximation of highly oscillatory integrals in both one dimension and higher dimensions to arbitrary precision?
4k views

### Is half precision supported by modern architecture?

I am new to computer science and I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is. I thought the 2008 revision of IEEE-754 ...
3k views

### Numerically stable way of computing angles between vectors

When applying the classical formula for the angle between two vectors: $$\alpha = \arccos \frac{\mathbf{v_1} \cdot \mathbf{v_2}}{\|\mathbf{v_1}\| \|\mathbf{v_2}\|}$$ one finds that, for very small/...
3k views

### How can I avoid catastrophic cancellation?

I have the following formula that I need to rewrite in order to avoid catastrophic cancellation. $$y =\sqrt{\frac{1}{2}\left(1-\sqrt{1-x^{2}}\right)}$$ As $x$ becomes smaller, $\sqrt{1-x^{2}}$ ...
904 views

### Scientific computing with Python with modern GPUs with double precision

Has anyone here used double precision scientific computing with new generation (e.g. K20) GPUs through Python? I know that this technology is rapidly evolving, but what is the best way to do this ...
4k views

### Single versus double floating-point precision

Single precision floating point numbers take up half the memory and on modern machines (even on GPUs it seems) operations can be done with them at almost twice the speed compared to double precision. ...
1k views

### Why can ill-conditioned linear systems be solved precisely?

According to the answer here, large condition number (for linear system solving) decreases the guaranteed number of correct digits in the floating point solution. Higher order differentiation matrices ...
2k views

### Numerical derivative and finite difference coefficients: any update of the Fornberg method?

When one want to compute numerical derivatives, the method presented by Bengt Fornberg here (and reported here) is very convenient (both precise and simple to implement). As the original paper date ...
885 views

### Higher precision floating-point arithmetic in numerical PDE

I have the impression, from very different resources and talks with researches, that there is a growing demand for high precision computations in numerical partial differential equations. Here, high ...
780 views

### Diagonalization of Dense Ill Conditioned Matrices

I am trying to diagonalize some dense, ill-conditioned matrices. In machine precision, results are inaccurate (returning negative eigenvalues, eigenvectors do not have the expected symmetries). I ...
403 views

### Small, unpredictable results in runs of a deterministic model

I have a sizable model (~5000 lines) written in C. It is a serial program, with no random number generation anywhere. It makes use of the FFTW library for functions using FFT - I do not know the ...
256 views

### Representing Eisenstein numbers without floats

I have a project where I need to use quadratic fields Specifically numbers of the form $a + b \sqrt{-3}$ with $a,b \in \mathbb{Q}$. For example here are the prime numbers in Eisenstein integers: I ...
457 views

### Updating matrix diagonal with Woodbury matrix identity and maintaining numerical accuracy

I have a dense matrix A and its corresponding inverse $A^{-1}$. The Woodbury matrix identity states: $$(A + UCV)^{-1} = A^{-1} - A^{-1}U(C^{-1} + VA^{-1}U)^{-1}VA^{-1}$$ I wish to perform small ...
139 views

### How can I numerically solve an ODE to $N$ provably correct digits?

Suppose we have an initial value problem of the form $$\frac{\mathrm{d} \mathbf{x}}{\mathrm{d} t} = f(\mathbf{x}) \qquad \mathbf{x}(0) = \mathbf{x}_0$$ where $\mathbf{x}_0 \in \mathbb{R}^n$ is known ...
657 views

### What algorithm for solving a set of stiff ODEs would be easiest to port to high precision floating point arithmetic?

I want to solve a relatively small system of stiff ODEs (< 10 first-order equations) using high precision floating point arithmetic (using MPFR or alike). What would be the easiest algorithm to ...
288 views

### Why should I renormalize physical variables?

I am working with legacy physical codes and I develop new ones based on the output of them. They all use their own internal normalization of variables (for example all distances are divided by the ...
3k views

### computing the determinant of a dense nonsymmetric 100x100 matrix having very big and very small eigenvalues

The motivation for my question is the following: in one of Project Euler questions there is a need to count the spanning trees of a rectangular grid graph of dimension 100x500. By the Matrix-Tree ...
110 views

### What good are hard-sphere event-driven molecular dynamics simulations in the face of chaos?

Simple hard-sphere dynamical systems can exhibit chaotic dynamics. Due to finite-precision arithmetic when implemented on a computer, the presence of chaos implies that for a given set of initial data,...
527 views

### What is the numerical difference between abs(z)^2 and z x z*, where z is a complex number

I am doing research in electromagnetics (a branch of physics) and I deal with complex numbers in my computing tasks (I use MATLAB). Let's say $z$ is a complex number. To calculate the square of the ...
493 views

### Does there exist an arbitrary-precision convex optimization solver?

I have a relatively simple convex optimization problem that involves less than 100 variables but contains a terribly ill-conditioned matrix. I have tried CVX and CPLEX; even though both can typically ...
162 views

### Numerically stable and fast sum of last K elements in sequence

Suppose I have a long, possibly infinite, sequence $x := [x_1, x_2, ...]$, and I want to use it to compute another sequence $y:=[y_1, y_2, ...]$ where each element is the sum of the last K elements of ...
793 views

2k views

### Determine the step size in a differential equation numerical solver

How can we define the precision we require in a numerical differential equation solver? What is it that I have to optimize to know? And how do I know that I'm at a sufficient time-step value? For ...
278 views

### Testing 1D root-finding procedures for robustness

How can I test whether a given 1D root-finding procedure is robust? I know that there are data sets and resources online for different kinds of optimization, but I have yet to find anything with ...
267 views

### precision loss in non-trigonometric, periodic functions using FFTW and NaNs after marching forward in time (Fortran)

I have developed a pseudospectral solver of the Navier-Stokes equations using FFTW. I tested my formulation of right hand sides (RHS) of the NS equations against standard trigonometric functions (...
373 views

### Known issues with eigenvalue numerics?

Are there any known issues (such as precision issues) with $\mathsf{MATLAB}$ eig and charpoly functions for large enough $\{-1,0,+1\}$ matrices? Even if I change $1$ or $2$ entries between matrices ...
278 views

### log-sum-exp trick for signed/complex numbers

I need to evaluate a sum of values that are on many different orders of magnitude in scale but might be signed. I’ve had great luck with the “log-sum-exp” trick for an unsigned version of my problem, ...
484 views

### GSL linear algebra LU/determinant precision

I am working with symmetric matrices of order $n \times n$ where $n \leq 50$. The diagonal elements of my matrices are a fixed number $d$ and the off diagonal elements are limited to two small numbers ...
158 views

### How to include negative number in the log-sum-exp?

I want to know summation of some small numbers, such as {e^-1000, -e^1001, e^1002...} If all numbers are positive, I can use log-sum-exp algorithm. But unfortunately, negative numbers are also ...
130 views

### Evaluating $\log(\exp(x)+1)$ for negative $x$

With double precision, I get $\log(\exp(-3)+1)=0.048587351573741958$, which already has $4$ incorrect digits, and $\log(\exp(-30)+1)=9.348... \times10^{-14}$, which only has two correct digits. What ...
Consider the following integral: $$S(q)=\int_{x=2}^q\sin^2\left(\frac{π\Gamma(x)}{2x}\right)dx$$ And consider the functions : $$R(q)=\frac{q}{\log(q)}$$ $$T(q)=\int_2^q\frac{1}{\log(x)}dx$$ I ...