Questions tagged [floating-point]

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

16 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9
votes
0answers
648 views

Implementing std::nextafter: Should denormals-are-zero mode affect it? If so, how?

This might be the wrong stackexchange site for this question. math.SE, cs.SE, programmers.SE, and of course stackoverflow are all possibilities. I'm hoping to reach an audience that might actually ...
5
votes
0answers
55 views

Evaluate Nth root of a rational to a correctly rounded float

Excuse my lack of vocabulary for I have no formal training in this field, which is also why I ask this question - it may be trivial or it may be impossible. I want to evaluate an expression in the ...
5
votes
0answers
297 views

Is there any catch on using `zgemm3m` vs regular `zgemm`?

I've just (to my embarrassment) encountered a BLAS-like extension of a matrix-matrix product subroutine gemm in Intel MKL: gemm3m...
5
votes
0answers
404 views

Any way to avoid catastrophic cancellation when computing the discriminant of a quadratic function?

Homework disclaimer... The task: We are using the following algorithm to solve the quadratic equation $x^2+2px+q=0$: $x_1=|p|+\sqrt{p^2-q}\mathtt{;}$ $\mathtt{if}\,p>0\,\mathtt{then}\,x_1=-x_1\...
5
votes
0answers
82 views

Comparing sum of floating points

I am currently working on a numerical algorithm involving a lot of floating point arithmetic, involving some badly conditioned problem sets. I am using the relation $|x - y| / (\max(|x|, |y|, 1)) \...
4
votes
0answers
76 views

Unstable Algorithms which become stable when hardware provides Kulisch exact dot product instruction

In John Gustaffson's book The End of Error, he discusses Ulrich Kulisch's exact dot product, which (in double precision) requires a 2100 bit fixed point register which rounds only once after the ...
3
votes
0answers
85 views

Stabilizing online average calculation

In Knuth, the following method for computing an average is presented: \begin{align*} M_{n} = M_{n-1} + (x_{n} - M_{n-1})/n \end{align*} (See here, if you don't have TAOCP.) Assuming the samples all ...
3
votes
0answers
84 views

What disables Gradual Overflow?

After studying the process of Gradual Underflow, I'm left a little curious as to why machines don't implement Gradual Overflow; where numbers exceeding the overflow level would be stored as un-...
3
votes
1answer
183 views

Rounding errors in images of Julia sets

One typically computes Julia sets by iterating a complex function, such as a polynomial or rational function. How do rounding errors affect the results? I'm looking for references on this issue, ...
2
votes
1answer
97 views

How to include negative number in the log-sum-exp?

I want to know summation of some small numbers, such as {e^-1000, -e^1001, e^1002...} If all numbers are positive, I can use log-sum-exp algorithm. But unfortunately, negative numbers are also ...
2
votes
0answers
92 views

Remez algorithm convergence

I have implemented the Remez algorithm in Python where all calculations were done with the Python mpmath library. I have noticed that sometimes the $|E_{max}|$ and $|E_{min}|$ do not monotonically ...
2
votes
0answers
49 views

Quick evaluation of floating point Absolute Error

I need to to find a quick and dirty way to estimate the absolute error introduced by a series of agebraic operations of IEEE single precision floating point numbers, a pessimistic result is ok. The ...
1
vote
0answers
55 views

Hardware supporting floats with fraction beyond 64 bit

Is there any computation accelerator (like a GPGPU) available, that natively (this means in hardware, not emulated by a library) supports arithmetics using floating point numbers with a fractional ...
1
vote
0answers
29 views

How is System.Decimal represented in memory bits?

I am trying to look at how different floating points are stored in memory. Firstly I looked at the System.Double (accessible by keyword ...
1
vote
0answers
131 views

Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
1
vote
0answers
96 views

Kahan Summation for Three-Term Recurrences

Kahan summation applies to summation problems, but not to three-term recurrence relations. However, a three-term recurrence shares many of the features of a summation-albeit with a rescaling step at ...