Questions tagged [floating-point]

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

Filter by
Sorted by
Tagged with
7
votes
1answer
209 views

Where does the floating point error come from? (Finite difference using matrix multiplication versus shifts and adding.)

In Julia it appears that one picks up some error terms when doing finite differences using matrix multiplication versus shifts and addition. ...
5
votes
3answers
161 views

Automatic finite differences

Given numbers $x, y \in \mathbb{R}$ where $$\frac{|y-x|}{|x|}$$ is small, and code that implements the function $f$ with a sequence of arithmetic operations, I would like to compute to high accuracy ...
4
votes
0answers
78 views

Unstable Algorithms which become stable when hardware provides Kulisch exact dot product instruction

In John Gustaffson's book The End of Error, he discusses Ulrich Kulisch's exact dot product, which (in double precision) requires a 2100 bit fixed point register which rounds only once after the ...
4
votes
2answers
3k views

Why are log and exp considered 'expensive' computations in ML?

In many resources/videos I see comments being made along the lines of "and we can see here that we have a logarithm/exponential so this will be an expensive computation to make." (such as ...
2
votes
1answer
104 views

How to include negative number in the log-sum-exp?

I want to know summation of some small numbers, such as {e^-1000, -e^1001, e^1002...} If all numbers are positive, I can use log-sum-exp algorithm. But unfortunately, negative numbers are also ...
0
votes
2answers
82 views

Validating that a code is a good spherical code

Apologies if this is a trivial question. If that is the case I imagine I would benefit from someone explaining where my understanding is lacking. I am having some trouble interpreting the (putatively ...
2
votes
1answer
66 views

Linearization of Remez algorithm rational case

In the rational case, we are interested to find polynomials $P(x)$ and $Q(x)$ s.t. $f(x_k)-P(x_k)/Q(x_k)=(-1)^kE$ for $k=1,2,\ldots, N$ where $N=deg(P)+deg(Q)+2$ This can be rewritten as $$ (1)~~~~~~(...
3
votes
1answer
98 views

How frequently scientific code uses comparisons NaN == NaN?

How frequently scientific code uses comparisons NaN == NaN? Reason of asking: from time to time compilers / software floating-point library implementations have ...
6
votes
3answers
2k views

Need for quad precision in scientific computing?

Even if quad precision is not directly supported by most CPUs, many Compilers (GNU, Intel) support them. Also some software packages allow to compile with quad precision, e.g. PETSc. But is there ...
4
votes
1answer
131 views

Floating point and global error in Euler Method

Inspired by this answer, I tried to understand when floating point errors come into visibility and to check it also comparing the plot of the numerical solution with Explicit Euler with the analytical ...
21
votes
4answers
72k views

How to determine the amount of FLOPs my computer is capable of

I would like to determine the theoretical number of FLOPs (Floating Point Operations) that my computer can do. Can someone please help me with this. (I would like to compare my computer to some ...
0
votes
1answer
68 views

What is the reason NVIDIA's Turing has twice the FP16 performance compared to it's FP32 whereas AMD has same performance in FP16 and FP32?

What is the reason NVIDIA's Turing has twice the FP16 performance compared to it's FP32 whereas AMD has same performance in FP16 and FP32? Like GTX 1650 Super has around 8 teraflops in FP16 but half ...
1
vote
0answers
55 views

Hardware supporting floats with fraction beyond 64 bit

Is there any computation accelerator (like a GPGPU) available, that natively (this means in hardware, not emulated by a library) supports arithmetics using floating point numbers with a fractional ...
2
votes
2answers
188 views

Hack for using hardware to take square roots of 128 bit numbers

I need to take integer square roots $\lfloor \sqrt{n}\rfloor$ of (lots of) 128 bit numbers $n$. Calling gmp seems to take surprisingly long (though I can't tell for sure, since gmp routines are not ...
2
votes
2answers
95 views

How can I detect lost of precision due to rounding in both floating point addition and multiplication?

From Computer Systems: a Programmer's Perspective: With single-precision floating point the expression (3.14+1e10)-1e10 evaluates to 0.0: the value 3.14 is lost ...
2
votes
0answers
95 views

Remez algorithm convergence

I have implemented the Remez algorithm in Python where all calculations were done with the Python mpmath library. I have noticed that sometimes the $|E_{max}|$ and $|E_{min}|$ do not monotonically ...
5
votes
1answer
119 views

Accurate and efficient computation of the logarithm of the ratio of two sines

For exploratory work related to special function implementations, I need to compute $\log \frac{\sin y}{\sin x} $, where $0 \le x \le y \le 2x < \frac{\pi}{2}$. Cases with $x \approx y$ in ...
8
votes
3answers
977 views

Accurate Polynomial Evaluation in Floating Point

What are the most accurate algorithms for evaluating a polynomial using floating point arithmetic? The internet seems to suggest that Horner's method is commonly used. In particular I have a cubic ...
1
vote
2answers
82 views

How do I speed up this function evaluation in matlab?

Half the run time of my code right now is evaluating a big function over many, many points, it takes maybe about 20 seconds per evaluation The function consists of a bunch of simple operations that ...
0
votes
1answer
71 views

Maximum lossless compression ratio for floating point time series

I want to compress an array of time series floating point data as much as possible. Currently the only algorithm I've found for this is XOR compression which works well, but doesn't compress the data ...
20
votes
4answers
4k views

Is half precision supported by modern architecture?

I am new to computer science and I was wondering whether half precision is supported by modern architecture in the same way as single or double precision is. I thought the 2008 revision of IEEE-754 ...
0
votes
1answer
146 views

Red flags for numerical computing?

I've learnt the hard way that you should avoid: computing small numbers as the difference of two large numbers evaluating chaotic functions with imprecise inputs. Are there any other red flags a ...
1
vote
1answer
74 views

Floating Point error when computing Binomial Distribution Probability

I have been given a binomial distribution: $$B(m+n;n,p)=\frac{(m+n)!}{m!n!}p^mq^n.$$ Here $m = 10^3$, $n=10^2$, $p=10^{-2}$, $q=1-p.$ I'm using MATLAB to compute log $B(m+n;n,p)$ and store the value ...
3
votes
1answer
131 views

Augmented arithmetic operations (IEEE-754-2019): output definition and implementation

In the new version of IEEE-754-2019: IEEE Standard for floating-point arithmetic, the augmented arithmetic operations were introduced. These operations can be particularly useful in certain numerical ...
0
votes
1answer
50 views

Convert decimal number in binary double precision, how to avoid the loss of the last digits after normalization?

I have the decimal number: $0.023$, and I want to convert in a binary number with $52$ bit of mantissa in Double Precision: if I go to convert, using this utility here, in non-normalized form, with $...
1
vote
0answers
30 views

How is System.Decimal represented in memory bits?

I am trying to look at how different floating points are stored in memory. Firstly I looked at the System.Double (accessible by keyword ...
7
votes
1answer
151 views

Numerically stable and fast sum of last K elements in sequence

Suppose I have a long, possibly infinite, sequence $x := [x_1, x_2, ...]$, and I want to use it to compute another sequence $y:=[y_1, y_2, ...]$ where each element is the sum of the last K elements of ...
1
vote
2answers
102 views

Matrix multiplication not working in Scilab

I entered an instruction to calculate the coordinates of a vector after a change of basis in order to repeat it many times with various vectors. X0=[1;1/2] is a ...
3
votes
1answer
268 views

log-sum-exp trick for signed/complex numbers

I need to evaluate a sum of values that are on many different orders of magnitude in scale but might be signed. I’ve had great luck with the “log-sum-exp” trick for an unsigned version of my problem, ...
2
votes
1answer
221 views

Numerical stability in the product of many matrices

I have to calculate in numpy the matrix-product of many matrices (~400). Are there common practices to increase numerical stability? If this is relevant, the matrices are $300\times 300$ orthogonal ...
0
votes
2answers
88 views

Is expm1 the right primitive?

I'm writing some code to calculate $\int_0^1 e^{ax} \mathrm{d} x$. Annoyingly there does not seem to be a way of doing this without if statements: ...
0
votes
2answers
182 views

How can I calculate the exponential integral?

(I originally asked this in a different exchange.) I'm writing a program that uses the prime-counting function. Right now, I'm using x/log(x), but I want to ...
4
votes
2answers
441 views

overflow upper incomplete gamma function

I want to calculate the following equation: $$\frac{\theta \Gamma \left(\kappa+1,\frac{o}{\theta }\right)-o \Gamma \left(\kappa,\frac{o}{\theta }\right)}{\Gamma (\kappa)}+o+s$$ with $s>0, o>0, ...
24
votes
10answers
7k views

Which algorithm is more accurate for computing the sum of a sorted array of numbers?

Given is an increasing finite sequence of positive numbers $z_{1} ,z_{2},.....z_{n}$. Which of the following two algorithms is better for computing the sum of the numbers? ...
15
votes
7answers
874 views

Robust computation of the mean of two numbers in floating-point?

Let x, y be two floating-point numbers. What's the right way to compute their mean? The naive way ...
5
votes
0answers
57 views

Evaluate Nth root of a rational to a correctly rounded float

Excuse my lack of vocabulary for I have no formal training in this field, which is also why I ask this question - it may be trivial or it may be impossible. I want to evaluate an expression in the ...
3
votes
1answer
77 views

Why are the round-off errors when solving the linear system $Ax = b$ of order $\varepsilon_\text{mach} x_j$?

I was reading a paper on arXiv where, in Section 2.4, the authors are discussing the error that arises in the solution of a linear system $$Ax = b,$$ or, to match up better with the paper, $$\Phi \...
5
votes
0answers
305 views

Is there any catch on using `zgemm3m` vs regular `zgemm`?

I've just (to my embarrassment) encountered a BLAS-like extension of a matrix-matrix product subroutine gemm in Intel MKL: gemm3m...
5
votes
2answers
114 views

Computing a ratio of exponential functions without overflow issues

I'm interested in computing pointwise values of the function $u(x) = \sinh(k-kx)/\sinh(k)$ for $x \in (0,1)$, where $k = 10^{4}$. A direct computation of course results in overflow issues due to the $\...
34
votes
2answers
11k views

When should log1p and expm1 be used?

I have a simple question that is really hard to Google (besides the canonical What Every Computer Scientist Should Know About Floating-Point Arithmetic paper). When should functions such as ...
6
votes
2answers
265 views

Is Highams' computation of mean worth the price?

In Accuracy and Stability of Numerical Algorithms, equation 1.6a, Higham gives the following update formula for the mean: $$ M_{1} := x_1, \quad M_{k+1} := M_{k} + \frac{x_k - M_k}{k} $$ Ok, one ...
4
votes
2answers
113 views

SLATEC Routine Computes Givens Rotation in Unexpected Way

Some Background I am working on a C++ translation of a SLATEC routine, R1UPDT, which performs a Givens rotation: $$r = \frac{1.0}{\sqrt{a^2 + b^2}}$$ Usually, ...
5
votes
0answers
410 views

Any way to avoid catastrophic cancellation when computing the discriminant of a quadratic function?

Homework disclaimer... The task: We are using the following algorithm to solve the quadratic equation $x^2+2px+q=0$: $x_1=|p|+\sqrt{p^2-q}\mathtt{;}$ $\mathtt{if}\,p>0\,\mathtt{then}\,x_1=-x_1\...
5
votes
0answers
82 views

Comparing sum of floating points

I am currently working on a numerical algorithm involving a lot of floating point arithmetic, involving some badly conditioned problem sets. I am using the relation $|x - y| / (\max(|x|, |y|, 1)) \...
1
vote
0answers
142 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
6
votes
2answers
2k views

Fast and Numerically Stable Pairwise Distance Algorithms

I'm looking for resources on fast, numerically stable pairwise euclidean distance algorithms. In particular, suppose $A \in \mathbb{R}^{M \times D}$ and $B \in \mathbb{R}^{N \times D}$ are two sets of ...
9
votes
2answers
4k views

Integer operations vs floating point operations

I have been working with an algorithm, which uses additions of floating point vectors, (sparse matrix of floats)x(dense vector of floats) dot products I recently found out that I can get the same ...
6
votes
1answer
213 views

Is it possible to proof a-b+b = a for all double floating-point numbers?

I want to know whether the equation : a-b+b = a is always true for a, b belongs to double precision floating-point number and |a|>=|b|. If the equation is true, how can I proof it? If not, what ...
6
votes
1answer
132 views

Stable computation of ratio of sums of large numbers

I have two sets of large positive numbers $a_1,\ldots,a_n$ and $b_1,\ldots,b_n$. By 'large' I mean of the order of $10^{10}$. I want to calculate the ratio $$R = \frac{a_1 - a_2 + \cdots +(-1)^{n+1}...
6
votes
2answers
291 views

Krylov subspace iterative methods in floating point arithmetic

Is there any work that considers Krylov subspace iterative methods in floating point arithmetic? I'm especially interested in how rounding errors influence the convergence and the accuracy of the ...