# Questions tagged [floating-point]

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

13 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
529 views

### Implementing std::nextafter: Should denormals-are-zero mode affect it? If so, how?

This might be the wrong stackexchange site for this question. math.SE, cs.SE, programmers.SE, and of course stackoverflow are all possibilities. I'm hoping to reach an audience that might actually ...
50 views

### Evaluate Nth root of a rational to a correctly rounded float

Excuse my lack of vocabulary for I have no formal training in this field, which is also why I ask this question - it may be trivial or it may be impossible. I want to evaluate an expression in the ...
236 views

84 views

### Stabilizing online average calculation

In Knuth, the following method for computing an average is presented: \begin{align*} M_{n} = M_{n-1} + (x_{n} - M_{n-1})/n \end{align*} (See here, if you don't have TAOCP.) Assuming the samples all ...
77 views

After studying the process of Gradual Underflow, I'm left a little curious as to why machines don't implement Gradual Overflow; where numbers exceeding the overflow level would be stored as un-...
173 views

### Rounding errors in images of Julia sets

One typically computes Julia sets by iterating a complex function, such as a polynomial or rational function. How do rounding errors affect the results? I'm looking for references on this issue, ...
46 views

### Quick evaluation of floating point Absolute Error

I need to to find a quick and dirty way to estimate the absolute error introduced by a series of agebraic operations of IEEE single precision floating point numbers, a pessimistic result is ok. The ...
25 views

### How is System.Decimal represented in memory bits?

I am trying to look at how different floating points are stored in memory. Firstly I looked at the System.Double (accessible by keyword ...
104 views

### Why is the method of im2col with GEMM is more efficient than the method of direction implementation with SIMD in CNN

The convolutional layers are most computationally intense parts of Convolutional neural networks (CNNs).Currently the common approach to impement convolutional layers is to expand the image into a ...
I'm reading the book "A First Course in Numerical Methods" by U. Ascher and C. Greif, and in the 2nd chapter it's written that ... we associate $x$ a floating point representation $fl(x)$ of the ...