Questions tagged [floating-point]

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

Filter by
Sorted by
Tagged with
20 votes
9 answers
4k views

C, Julia, Python, Maxima, Mathematica, ChatGPT and numerical errors

I am completely stunned how numerical errors can diverge for so innocent programs. In Python 3.11.7 the program ...
Smilia's user avatar
  • 468
0 votes
2 answers
113 views

Is computing the 2-norm of a vector numerically stable?

Is computing the 2-norm of a vector $x$, computed by setting $\alpha = \max_i x_i$ and then computing $|\alpha| \cdot ||x / \alpha ||_2$ (e.g., as in this link) numerically stable? There are two parts ...
i901234's user avatar
  • 101
1 vote
0 answers
70 views

Has the arithmetic for exotic (unsigned float with positive exponent) number format been solved?

The data type is a doubly unsigned float. This is where the value and exponent are both strictly positive. The range of this number should include $0$ and $[1, ~2^{\text{exponent}})$, skipping all ...
Jeff's user avatar
  • 133
2 votes
1 answer
118 views

The name of an exotic number format: float with exponent bits replaced with another float

What is the name of the numeric data type where a float has its exponent bits replaced with another floating point value to use as the outer float's exponent? This is a tip-of-the-tongue problem, as I ...
Jeff's user avatar
  • 133
2 votes
2 answers
152 views

How to design a sin and an arcsin function such that arcsin(sin(x))=x, where x is a finite precision floating point number

As commonly known for programming on computer, if x is a finite precision floating-point number such as double/float in C language, arcsin(sin(x)) is usually not equal to x due to the numerical issue. ...
user3677630's user avatar
4 votes
1 answer
206 views

Numerical accuracy of expression involving norm squared

I am computing the following quantity: $$ \text{lhs} := ||a+b||^2 = ||a||^2 + 2a^\top b + ||b||^2 =: \text{rhs} $$ for $a=c-d$, where $a,b,c,d$ are $n$-vectors. Is there a rule of thumb for when I ...
jjjjjj's user avatar
  • 325
3 votes
1 answer
98 views

SVD testing non zero values

I was looking at the matlab function pinv.m for the compuation of the pseudoinverse. The code uses the singular values decomposition. $$ A = U D V $$ When looking for non-zero diagonal elements it ...
pinpon's user avatar
  • 153
3 votes
1 answer
73 views

Dynamic tolerance in a conditional loop to obtain maximum precision allowed by machine floating point numbers

I have coded a simple program for a root finding problem using Halley's method. Here is the code: ...
Hosein Javanmardi's user avatar
6 votes
2 answers
145 views

Method to compute $a^n - b^n$

Given two floating point numbers $a,b$ with $a > b$ and an integer $n$, what is the most accurate way to compute $$ a^n - b^n $$ ? We can assume both $a,b$ are between 1 and 2. Lets assume both $a^...
vibe's user avatar
  • 1,048
1 vote
1 answer
47 views

robustness of geometric predicates in Euclidean vs homogeneous coordinates

The signed volume of the triangle formed by the points $p, q, r$ in the plane is defined to be $$\text{volume}(p, q, r) \equiv \det\left[\begin{matrix}q_1 - p_1 & r_1 - p_1 \\ q_2 - p_2 & r_2 -...
Daniel Shapero's user avatar
4 votes
1 answer
119 views

Summation of trigonometric functions results in error with finite precision

Consider the following expression: $$f(t) = B+\sum_{k=1}^{N} A_k\cos(\omega_kt)$$ where $A$ and $B$ are known. the frequencies are also known but are not multiples of a fundamental frequency. However, ...
Hosein Javanmardi's user avatar
8 votes
1 answer
220 views

Accurate computation of logbinomial(x,y)

I need to compute the logarithm of the binomial coefficient, $$\log\binom{x}{y} = -\log\mathrm{B}(y + 1, x - y + 1) - \log(x + 1)$$ accurately, where $\mathrm{B}(x,y)$ is the Beta function. I'm aware ...
a06e's user avatar
  • 1,729
0 votes
1 answer
114 views

Robust unit test for reciprocal approximation

Let $x$ and $y$ be representable floating point numbers. I'm looking for a unit test which can ensure that my user's compiler has not made the reciprocal approximation $\mathrm{fl}(x/y) \approx \...
user14717's user avatar
  • 2,155
2 votes
1 answer
104 views

Padding length and error analysis of discrete convolution by FFT

The standard algorithm for discrete convolution of two vectors $x\in \mathbb{R}^{n}$ and $y \in \mathbb{R}^{m}$ is (in essence) a FFT of the two input vectors, multiplication of the two elementwise, ...
user14717's user avatar
  • 2,155
4 votes
0 answers
84 views

Numerical algorithms made stable by unums which are unstable on IEEE floats

For unums, there is good evidence (see figure 5) that accuracy is better than IEEE floats. (Note: I use the term "unum" broadly to refer to any of the various iterations and revisions to the ...
user14717's user avatar
  • 2,155
0 votes
2 answers
340 views

float in C: Size of the exponent

This question is very related to the question Number of decimal of float and double in C. In the second table we may see that the exponent of the float are from -38 to +38. But the IEEE754 standard ...
yemino's user avatar
  • 515
2 votes
1 answer
190 views

Accuracy loss in single-precision Euclidean norm computation

I do hydrodynamics simulations with Fortran and recently I met with this issue: I have a single-precision array b of length ...
H. Zhou's user avatar
  • 123
3 votes
2 answers
4k views

GPU vs CPU FLOP counts

I apologise if this is somewhat of a rookie question. So, from my understanding, on a GPU board, far more of the space is allocated to ALUs compared to CPUs which have far more cache available. This ...
Enforce's user avatar
  • 141
6 votes
1 answer
526 views

Polynomial approximation for floating-point arithmetic

I cannot remember where I picked this up, but during my time reading about polynomial approximation for floating-point arithmetic of sin(x), I vaguely remember that ...
Quang Thinh Ha's user avatar
17 votes
4 answers
4k views

What are some good strategies to test a floating point arithmetic implementation for double numbers?

For IEEE, the single representation is 1-bit sign, 8-bit exponent and 23-bit mantissa. This means that at each exponent value, you can test all 2^23-1 (roughly 9mil cases) possible combination of ...
Quang Thinh Ha's user avatar
3 votes
1 answer
389 views

How to interpret if $\displaystyle \sum_{j = 0}^{n} \frac {1}{j!}$ is a stable algorithm for computing $e$?

I am trying to solve problem $15.1$ from Numerical Linear Algebra by Trefethen and Bau, which reads Determine whether the algorithm is backward stable, stable but not backward stable, or unstable. ...
user56202's user avatar
  • 163
2 votes
1 answer
845 views

Why is subtraction a stable operation?

In Numerical Linear Algebra by Trefethen & Bau, it is claimed that subtraction is backward stable. Here is the proof: Let $f(x, y) = x-y$ and let $\tilde f(x,y)$ be the answer you get when doing $...
user56202's user avatar
  • 163
1 vote
0 answers
73 views

Calculating the number of Flops of SPH density calculation

I would like to calculate the number of floating point operations (Flops) my code is performing in my machine. To do so, I would like to be sure I am counting the operations in the inner-most loop ...
Hydro Guy's user avatar
  • 121
13 votes
1 answer
6k views

Meaning of "-0.0" in Python?

We are finding in Python some occasional errors in our coordinate transforms and other similar computations that produce a result of -0.0. What purpose does this ...
Chris Ison's user avatar
5 votes
3 answers
238 views

Automatic finite differences

Given numbers $x, y \in \mathbb{R}$ where $$\frac{|y-x|}{|x|}$$ is small, and code that implements the function $f$ with a sequence of arithmetic operations, I would like to compute to high accuracy ...
Federico Poloni's user avatar
4 votes
0 answers
91 views

Unstable Algorithms which become stable when hardware provides Kulisch exact dot product instruction

In John Gustaffson's book The End of Error, he discusses Ulrich Kulisch's exact dot product, which (in double precision) requires a 2100 bit fixed point register which rounds only once after the ...
user14717's user avatar
  • 2,155
7 votes
2 answers
7k views

Why are log and exp considered 'expensive' computations in ML?

In many resources/videos I see comments being made along the lines of "and we can see here that we have a logarithm/exponential so this will be an expensive computation to make." (such as ...
DChaps's user avatar
  • 173
0 votes
2 answers
95 views

Validating that a code is a good spherical code

Apologies if this is a trivial question. If that is the case I imagine I would benefit from someone explaining where my understanding is lacking. I am having some trouble interpreting the (putatively ...
Martin C.'s user avatar
  • 229
3 votes
1 answer
104 views

How frequently scientific code uses comparisons NaN == NaN?

How frequently scientific code uses comparisons NaN == NaN? Reason of asking: from time to time compilers / software floating-point library implementations have ...
pmor's user avatar
  • 131
5 votes
1 answer
406 views

Floating point and global error in Euler Method

Inspired by this answer, I tried to understand when floating point errors come into visibility and to check it also comparing the plot of the numerical solution with Explicit Euler with the analytical ...
Vefhug's user avatar
  • 309
3 votes
1 answer
408 views

How to include negative number in the log-sum-exp?

I want to know summation of some small numbers, such as {e^-1000, -e^1001, e^1002...} If all numbers are positive, I can use log-sum-exp algorithm. But unfortunately, negative numbers are also ...
jasson31's user avatar
1 vote
0 answers
60 views

Hardware supporting floats with fraction beyond 64 bit

Is there any computation accelerator (like a GPGPU) available, that natively (this means in hardware, not emulated by a library) supports arithmetics using floating point numbers with a fractional ...
Silicomancer's user avatar
0 votes
1 answer
190 views

What is the reason NVIDIA's Turing has twice the FP16 performance compared to it's FP32 whereas AMD has same performance in FP16 and FP32?

What is the reason NVIDIA's Turing has twice the FP16 performance compared to it's FP32 whereas AMD has same performance in FP16 and FP32? Like GTX 1650 Super has around 8 teraflops in FP16 but half ...
noviceFedora's user avatar
3 votes
2 answers
905 views

Hack for using hardware to take square roots of 128 bit numbers

I need to take integer square roots $\lfloor \sqrt{n}\rfloor$ of (lots of) 128 bit numbers $n$. Calling gmp seems to take surprisingly long (though I can't tell for sure, since gmp routines are not ...
H A Helfgott's user avatar
2 votes
2 answers
184 views

How can I detect lost of precision due to rounding in both floating point addition and multiplication?

From Computer Systems: a Programmer's Perspective: With single-precision floating point the expression (3.14+1e10)-1e10 evaluates to 0.0: the value 3.14 is lost ...
Tim's user avatar
  • 1,281
3 votes
1 answer
111 views

Linearization of Remez algorithm rational case

In the rational case, we are interested to find polynomials $P(x)$ and $Q(x)$ s.t. $f(x_k)-P(x_k)/Q(x_k)=(-1)^kE$ for $k=1,2,\ldots, N$ where $N=deg(P)+deg(Q)+2$ This can be rewritten as $$ (1)~~~~~~(...
Daniel's user avatar
  • 181
3 votes
0 answers
202 views

Remez algorithm convergence

I have implemented the Remez algorithm in Python where all calculations were done with the Python mpmath library. I have noticed that sometimes the $|E_{max}|$ and $|E_{min}|$ do not monotonically ...
Daniel's user avatar
  • 181
5 votes
1 answer
145 views

Accurate and efficient computation of the logarithm of the ratio of two sines

For exploratory work related to special function implementations, I need to compute $\log \frac{\sin y}{\sin x} $, where $0 \le x \le y \le 2x < \frac{\pi}{2}$. Cases with $x \approx y$ in ...
njuffa's user avatar
  • 1,865
1 vote
2 answers
126 views

How do I speed up this function evaluation in matlab?

Half the run time of my code right now is evaluating a big function over many, many points, it takes maybe about 20 seconds per evaluation The function consists of a bunch of simple operations that ...
Sam Christensen's user avatar
2 votes
1 answer
242 views

Maximum lossless compression ratio for floating point time series

I want to compress an array of time series floating point data as much as possible. Currently the only algorithm I've found for this is XOR compression which works well, but doesn't compress the data ...
Marcel's user avatar
  • 131
0 votes
1 answer
164 views

Red flags for numerical computing?

I've learnt the hard way that you should avoid: computing small numbers as the difference of two large numbers evaluating chaotic functions with imprecise inputs. Are there any other red flags a ...
Tom Huntington's user avatar
1 vote
1 answer
142 views

Floating Point error when computing Binomial Distribution Probability

I have been given a binomial distribution: $$B(m+n;n,p)=\frac{(m+n)!}{m!n!}p^mq^n.$$ Here $m = 10^3$, $n=10^2$, $p=10^{-2}$, $q=1-p.$ I'm using MATLAB to compute log $B(m+n;n,p)$ and store the value ...
user avatar
0 votes
1 answer
93 views

Convert decimal number in binary double precision, how to avoid the loss of the last digits after normalization?

I have the decimal number: $0.023$, and I want to convert in a binary number with $52$ bit of mantissa in Double Precision: if I go to convert, using this utility here, in non-normalized form, with $...
JB-Franco's user avatar
  • 137
3 votes
1 answer
250 views

Augmented arithmetic operations (IEEE-754-2019): output definition and implementation

In the new version of IEEE-754-2019: IEEE Standard for floating-point arithmetic, the augmented arithmetic operations were introduced. These operations can be particularly useful in certain numerical ...
Anton Menshov's user avatar
  • 8,652
30 votes
4 answers
8k views

Is half precision supported by modern architecture?

I am new to computer science and I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is. I thought the 2008 revision of IEEE-754 ...
Asad Mehasi's user avatar
1 vote
0 answers
41 views

How is System.Decimal represented in memory bits?

I am trying to look at how different floating points are stored in memory. Firstly I looked at the System.Double (accessible by keyword ...
user13892's user avatar
  • 119
7 votes
1 answer
252 views

Numerically stable and fast sum of last K elements in sequence

Suppose I have a long, possibly infinite, sequence $x := [x_1, x_2, ...]$, and I want to use it to compute another sequence $y:=[y_1, y_2, ...]$ where each element is the sum of the last K elements of ...
Peter's user avatar
  • 171
3 votes
1 answer
565 views

log-sum-exp trick for signed/complex numbers

I need to evaluate a sum of values that are on many different orders of magnitude in scale but might be signed. I’ve had great luck with the “log-sum-exp” trick for an unsigned version of my problem, ...
Justin Solomon's user avatar
2 votes
1 answer
701 views

Numerical stability in the product of many matrices

I have to calculate in numpy the matrix-product of many matrices (~400). Are there common practices to increase numerical stability? If this is relevant, the matrices are $300\times 300$ orthogonal ...
user1767774's user avatar
0 votes
2 answers
114 views

Is expm1 the right primitive?

I'm writing some code to calculate $\int_0^1 e^{ax} \mathrm{d} x$. Annoyingly there does not seem to be a way of doing this without if statements: ...
user357269's user avatar