Skip to main content

Questions tagged [precision]

Issues related to the representation of numerical quantities in a finite representation in a given base differing from their exact mathematical value.

Filter by
Sorted by
Tagged with
7 votes
1 answer
180 views

Unexpected result when summing sorted (and unsorted) positive floating point numbers

I am exploring Higham's excellent Accuracy and Stability of Numerical Algorithms and chapter 4 is dedicated to summation. So I decided to test the most basic thing. Summing positive random numbers ...
lucmobz's user avatar
  • 71
5 votes
0 answers
129 views

Single precision vs double precision conjugate gradients

I tested my conjugate gradients implementation with float and double precision and contrary to my guess the double code was twice faster than the single precision code. The reason is that I need many ...
lightxbulb's user avatar
  • 2,122
2 votes
0 answers
99 views

Runge Kutta 4th order: unexpected result

My problem in brief: in some situations, the Runge Kutta 4th order method (RK4) doesn't seem to give 4th order improvement when using a smaller time step. I wonder how this worse-than-expected result ...
gamma1954's user avatar
  • 121
1 vote
1 answer
522 views

Float equality tolerance for single and half precision

Suppose the metric is abs(a-b) <= rtol * max(abs(a), abs(b)) i.e. math.isclose with ...
OverLordGoldDragon's user avatar
2 votes
1 answer
172 views

Numerically stable way to implement Cramer's rule analog

Problem statement Let $A$ be an $n\times n$ matrix and $b$ an $n$-dimensional vector. For $j\in \{1, \dots, n \}$, let $A_j$ be the matrix where we take $A$ and replace the $j^{\rm th}$ column with $b$...
Joe's user avatar
  • 31
3 votes
1 answer
76 views

Dynamic tolerance in a conditional loop to obtain maximum precision allowed by machine floating point numbers

I have coded a simple program for a root finding problem using Halley's method. Here is the code: ...
Hosein Javanmardi's user avatar
4 votes
1 answer
119 views

Summation of trigonometric functions results in error with finite precision

Consider the following expression: $$f(t) = B+\sum_{k=1}^{N} A_k\cos(\omega_kt)$$ where $A$ and $B$ are known. the frequencies are also known but are not multiples of a fundamental frequency. However, ...
Hosein Javanmardi's user avatar
0 votes
1 answer
117 views

Robust unit test for reciprocal approximation

Let $x$ and $y$ be representable floating point numbers. I'm looking for a unit test which can ensure that my user's compiler has not made the reciprocal approximation $\mathrm{fl}(x/y) \approx \...
user14717's user avatar
  • 2,155
1 vote
1 answer
85 views

What are the Exact Rules for Significant Figures, Precision, and Uncertainty?

In the physical sciences (which are physics, chemistry, astronomy, materials science, etc.), we learned that the uncertainty is +/- the smallest unit (which is 1) of the last significant figure if the ...
CoastCity Lapse 00crashtest's user avatar
2 votes
1 answer
195 views

Accuracy loss in single-precision Euclidean norm computation

I do hydrodynamics simulations with Fortran and recently I met with this issue: I have a single-precision array b of length ...
H. Zhou's user avatar
  • 123
2 votes
1 answer
407 views

High precision numerical integration of discrete data with Matlab

I have discrete data of a function plotted below: The "Y" values of the function near "X=1.57" are very close to each other and zero, like 9.25558265263186E-11 and 5....
tio's user avatar
  • 23
2 votes
0 answers
376 views

Efficient way of calculating a cumulative integral with prefactor

I have a grid of points $x_i$ and corresponding function values $y_i=f(x_i)$. I'm interesting in something like the cumulant of $f$, but it has an awkward prefactor. The desired quantity we'll call $$...
Root of All Things's user avatar
0 votes
1 answer
142 views

How error scales with numerical precision in molecular dynamics?

In terms of time-step, numerical error in molecular dynamics scales with square, i.e. $error \approx dt^2$. But how it look for numerical precision ? E.g. how much bigger will be numerical error when ...
Daniel Wiczew's user avatar
3 votes
1 answer
419 views

How to include negative number in the log-sum-exp?

I want to know summation of some small numbers, such as {e^-1000, -e^1001, e^1002...} If all numbers are positive, I can use log-sum-exp algorithm. But unfortunately, negative numbers are also ...
jasson31's user avatar
1 vote
0 answers
60 views

Hardware supporting floats with fraction beyond 64 bit

Is there any computation accelerator (like a GPGPU) available, that natively (this means in hardware, not emulated by a library) supports arithmetics using floating point numbers with a fractional ...
Silicomancer's user avatar
2 votes
2 answers
184 views

How can I detect lost of precision due to rounding in both floating point addition and multiplication?

From Computer Systems: a Programmer's Perspective: With single-precision floating point the expression (3.14+1e10)-1e10 evaluates to 0.0: the value 3.14 is lost ...
Tim's user avatar
  • 1,281
17 votes
1 answer
3k views

How can I avoid catastrophic cancellation?

I have the following formula that I need to rewrite in order to avoid catastrophic cancellation. $$y =\sqrt{\frac{1}{2}\left(1-\sqrt{1-x^{2}}\right)}$$ As $x$ becomes smaller, $\sqrt{1-x^{2}}$ ...
Tim's user avatar
  • 273
0 votes
1 answer
164 views

Red flags for numerical computing?

I've learnt the hard way that you should avoid: computing small numbers as the difference of two large numbers evaluating chaotic functions with imprecise inputs. Are there any other red flags a ...
Tom Huntington's user avatar
0 votes
1 answer
94 views

Convert decimal number in binary double precision, how to avoid the loss of the last digits after normalization?

I have the decimal number: $0.023$, and I want to convert in a binary number with $52$ bit of mantissa in Double Precision: if I go to convert, using this utility here, in non-normalized form, with $...
JB-Franco's user avatar
  • 137
30 votes
4 answers
8k views

Is half precision supported by modern architecture?

I am new to computer science and I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is. I thought the 2008 revision of IEEE-754 ...
Asad Mehasi's user avatar
0 votes
0 answers
128 views

Division and Modulus for Large DIVISORS

Excel and also Excel VBA have no built-in support for arbitrary precision arithmetic. There are a few very large add-ins that can be installed to do these sorts of calculations where the operands are ...
Xyrph's user avatar
  • 9
1 vote
3 answers
289 views

Comparison of integrals with a function:

Consider the following integral: $$S(q)=\int_{x=2}^q\sin^2\left(\frac{π\Gamma(x)}{2x}\right)dx$$ And consider the functions : $$R(q)=\frac{q}{\log(q)}$$ $$T(q)=\int_2^q\frac{1}{\log(x)}dx$$ I ...
bambi's user avatar
  • 119
1 vote
1 answer
377 views

Find precision, or the number of digits in the mantissa, in a floating point machine number

We have that: $\epsilon$ is the smallest positive machine number that summed to $1$ resuts in $(\epsilon + 1)$: i.e. the smallest number greater than $1$: if $p$ is precision and $\beta$ the base: $$...
JB-Franco's user avatar
  • 137
7 votes
1 answer
254 views

Numerically stable and fast sum of last K elements in sequence

Suppose I have a long, possibly infinite, sequence $x := [x_1, x_2, ...]$, and I want to use it to compute another sequence $y:=[y_1, y_2, ...]$ where each element is the sum of the last K elements of ...
Peter's user avatar
  • 171
2 votes
1 answer
351 views

Best way to check if SOR solution has converged for 2d matrix

I have written a SOR algorithm to solve the Laplace equation on a 2d grid. The outside of the grid is fixed at 0 and the central square is fixed at 10. I can obtain the fully converged solution for ...
user8384493's user avatar
3 votes
1 answer
590 views

log-sum-exp trick for signed/complex numbers

I need to evaluate a sum of values that are on many different orders of magnitude in scale but might be signed. I’ve had great luck with the “log-sum-exp” trick for an unsigned version of my problem, ...
Justin Solomon's user avatar
5 votes
1 answer
236 views

How do I globally change the precision of a piece of code in Python to debug it?

I am solving a system of non-linear equations using the Newton-Raphson method in Python. This involves using the solve(Ax,b) function (...
Abel Thayil's user avatar
2 votes
1 answer
724 views

Numerical stability in the product of many matrices

I have to calculate in numpy the matrix-product of many matrices (~400). Are there common practices to increase numerical stability? If this is relevant, the matrices are $300\times 300$ orthogonal ...
user1767774's user avatar
2 votes
1 answer
538 views

How to perform an eigendecomposition of a general complex matrix with arbitrary precision in C/C++

I need to obtain the Eigenvectors of a general complex matrix, but with quadruple precision. Is anyone aware of a means to do this? I currently use Tux Eigen, and I see that in their unsupported ...
AlexD's user avatar
  • 131
4 votes
3 answers
1k views

Inverse of ill-conditioned symmetric matrix

I've got a matrix K, with dimensions $(n, n)$ where each element is computed using the following equation: $$K_{i, j} = \exp(-\alpha t_i^2 -\gamma(t_i - t_j)^2 - \...
adityar's user avatar
  • 143
5 votes
0 answers
64 views

Evaluate Nth root of a rational to a correctly rounded float

Excuse my lack of vocabulary for I have no formal training in this field, which is also why I ask this question - it may be trivial or it may be impossible. I want to evaluate an expression in the ...
pipe's user avatar
  • 153
1 vote
1 answer
786 views

Conjugate gradient - ill-conditioning and numerical tolerance

I would like to solve system $Ax=b$, where $A$ is SPD, but very ill-conditioned ($\text{cond}(A)>10^{11}$). I am interested in using UNpreconditioned version of the conjugate gradient method. Is ...
computational_scientist's user avatar
4 votes
2 answers
121 views

Analytical convergent sequence and numerical divergent sequence

Is it possible to construct a sequence that converges in theory but when computed numerically with a computer program is diverging. I feel that today our computer programs doesn't allow such ...
Smilia's user avatar
  • 468
1 vote
0 answers
227 views

How to compute large condition number of a matrix in Python?

I have a matrix that is extremely singular, but I am still interested in computing the exact condition number, which is the ratio between the largest and smallest singular values. Is it possible to ...
ArtificiallyIntelligent's user avatar
6 votes
1 answer
170 views

Compile-time error control vs. interval arithmetic?

I use interval arithmetic for reliable computing. Now, a procedure coded in a good implementation of interval arithmetic takes perhaps about eight times as much as the same procedure carried out ...
H A Helfgott's user avatar
6 votes
1 answer
251 views

Is it possible to proof a-b+b = a for all double floating-point numbers?

I want to know whether the equation : a-b+b = a is always true for a, b belongs to double precision floating-point number and |a|>=|b|. If the equation is true, how can I proof it? If not, what ...
Star's user avatar
  • 63
7 votes
2 answers
150 views

What good are hard-sphere event-driven molecular dynamics simulations in the face of chaos?

Simple hard-sphere dynamical systems can exhibit chaotic dynamics. Due to finite-precision arithmetic when implemented on a computer, the presence of chaos implies that for a given set of initial data,...
joshphysics's user avatar
0 votes
1 answer
48 views

Time iteration no longer smooth after using scaled units

I have a time iteration function looked on a 2D surface like this. Since the numbers wee very small i.e. hbar=6.6260700404e-34./(2*pi), my professor told me to use our own "scaled unites" during the ...
J C's user avatar
  • 153
2 votes
2 answers
172 views

Evaluating $\log(\exp(x)+1)$ for negative $x$

With double precision, I get $\log(\exp(-3)+1)=0.048587351573741958$, which already has $4$ incorrect digits, and $\log(\exp(-30)+1)=9.348... \times10^{-14}$, which only has two correct digits. What ...
Bananach's user avatar
  • 799
0 votes
1 answer
469 views

High precision Discrete Fourier Transform in c

I'm trying to do a high precision discrete fourier transform on a signal. To examine the precision, I use a gaussian function as the signal, because the fourier transform is also a gaussian function. ...
Roy Liao's user avatar
13 votes
2 answers
2k views

Why can ill-conditioned linear systems be solved precisely?

According to the answer here, large condition number (for linear system solving) decreases the guaranteed number of correct digits in the floating point solution. Higher order differentiation matrices ...
Zoltan Csati's user avatar
1 vote
0 answers
218 views

Numerical Precision in Matrix Inversion Routines

Let's say I have a matrix $\mathbf{A}\in\mathbb{R}^{n\times n}$, and its associated inverse, $\mathbf{A}^{-1}$. The elements of $\mathbf{A}$ are given in IEEE single precision, i.e. 23-bit mantissa, ...
Homer Simpson's user avatar
21 votes
2 answers
5k views

Numerically stable way of computing angles between vectors

When applying the classical formula for the angle between two vectors: $$\alpha = \arccos \frac{\mathbf{v_1} \cdot \mathbf{v_2}}{\|\mathbf{v_1}\| \|\mathbf{v_2}\|}$$ one finds that, for very small/...
astrojuanlu's user avatar
  • 1,187
1 vote
0 answers
81 views

Computing the change of function at two close points without cancellation

I want to compute the difference $\Delta f(x_1,x_2) = f(x_1)-f(x_2)$ of a smooth function $f(x)$ at two points $x_1$ and $x_2$ which are close to each other. The magnitude of the expected result, $|\...
norio's user avatar
  • 153
5 votes
1 answer
1k views

Half precision in Fortran

To improve the time efficiency of my code, I'd like to test a lower precision for real number, using e.g. half precision (2 bytes). However, I'm not sure if I can do that in Fortran. After playing ...
Matthieu's user avatar
7 votes
1 answer
213 views

Stable computation of ratio of sums of large numbers

I have two sets of large positive numbers $a_1,\ldots,a_n$ and $b_1,\ldots,b_n$. By 'large' I mean of the order of $10^{10}$. I want to calculate the ratio $$R = \frac{a_1 - a_2 + \cdots +(-1)^{n+1}...
EpsilonDelta's user avatar
1 vote
2 answers
172 views

How to deal with big numbers in intermediate calculations?

I have a rather long expression (https://pastebin.com/jUsxdCCs) that is an analytical solution of a set of differential equations generated symbolically from Maple. I need to solve a set of equations ...
Chintan Pathak's user avatar
0 votes
1 answer
209 views

How to deal with very low numerical values in C?

I have to work with values such as 1e-15 in my code but I can't. Indeed, because of low precision these values are equivalent to 0. Any ideas?
T. Auerrac's user avatar
2 votes
2 answers
567 views

Precision loss in Matrix-Vector product when applying Finite-Difference scheme

I am applying a 6th order Finite-Difference differentiation scheme as seen in http://www.scholarpedia.org/article/Method_of_lines/example_implementation/dss006 There seems to be severe numerical/...
Dr Krishnakumar Gopalakrishnan's user avatar
1 vote
1 answer
430 views

Any FOSS MATLAB/Octave toolbox for high-speed variable precision arithmetic?

I need to use variable precision arithmetic in MATLAB for an expensive set of computation. The vpa function provided by the symbolic math toolbox is very slow. I found a non-free alternative toolbox ...
Dr Krishnakumar Gopalakrishnan's user avatar