# Tagged Questions

A method of representing numbers by a fixed number of significant digits, and the exponent of some base number. They are characterized in the form ${(significant digits)}*base^{exponent}$. Typically, numbers are represented with respect to base = 2 (binary).

167 views

### Small, unpredictable results in runs of a deterministic model

I have a sizable model (~5000 lines) written in C. It is a serial program, with no random number generation anywhere. It makes use of the FFTW library for functions using FFT - I do not know the ...
131 views

### Fortran round-off error with floating point operations

I have simple code, which flags nodes with in region enclosed by cylinder. On implementing the code, the result is mild tilt of the cylinder observed case with $\theta=90^{\circ}$. The algorithm for ...
86 views

### Where does the floating point error come from? (Finite difference using matrix multiplication versus shifts and adding.)

In Julia it appears that one picks up some error terms when doing finite differences using matrix multiplication versus shifts and addition. ...
54 views

### Numerical computation of the complex elliptic integral $E(k)$ for medium $|k|$

I have implemented Carlson's algorithm for $E(k)$ from Numerical computation of real or complex elliptic integrals (available from ArXiv eprint, see also DLMF). It is essentially his formula (46) ...
98 views

### Stabilizing a 3x3 real symmetric matrix eigenvalue calculation

I have many 3x3 real symmetric matrices for which I need to determine the eigenvalues. Wikipedia gives a nice non-iterative algorithm for this case, which I have translated into C++: ...
125 views

### Accurate computation of the current time in time integrator

I implement Runge--Kutta method for time integration of the system of nonlinear conservation laws $$u_t + f(u)_x = 0.$$ As the system is nonlinear, we have to recompute time step ...
41 views

### Why the greatest exponent that can be represented in single-precision floating-pointer numbers is 127 (and not 128)?

I was told by a class mate that the smallest exponent that we can represent by a single-precision floating-point number (which uses 8 bits for the exponent) is $-126$ and the greatest is $127$. I ...
16 views

### t digits are used to represent the mantissa in floating-point system, but rounding unit is calculated for doubles with 53 bits

I'm reading the book "A First Course in Numerical Methods" by U. Ascher and C. Greif, and in the 2nd chapter it's written that ... we associate $x$ a floating point representation $fl(x)$ of the ...
49 views

### Is more robust to digit cancellation the sum operator or the mean operator?

Note: this question is not strictly related to Matlab or any other environment (even though I would prefer that you refer to my Matlab code). I've n = 10000 ...
39 views

### How to compute $\mathrm{proj}_{SDP}(C\odot X)./C$ without numerical problems?

I have a matrix, $X$, it is symmetric. I project $C \odot X$ and $D\odot X$ to semidefinite cone. $C$ is a Gramian matrix with some elements near zero and of course semidefinite, with one row and ...
98 views

### Addition and subtraction of two floats in Python

Yesterday I was wondering how floats are handled in a computer and what they look like in binary... I learnt about the single-precision floating-point and I tried to see the limit of that format... I ...
64 views

### Implementing std::nextafter: Should denormals-are-zero mode affect it? If so, how?

This might be the wrong stackexchange site for this question. math.SE, cs.SE, programmers.SE, and of course stackoverflow are all possibilities. I'm hoping to reach an audience that might actually ...
321 views

### Hardware performance, floating point functions

First of all, hope I've found the right forum for this question, if I haven't please pass me on to a one which would fit better. Out of curiosity from an argument with someone who may or may not be ...
90 views

203 views

### Why should I renormalize physical variables?

I am working with legacy physical codes and I develop new ones based on the output of them. They all use their own internal normalization of variables (for example all distances are divided by the ...
1k views

### How to avoid catastrophic cancellation in python function?

I am having trouble implementing a function numerically. It suffers from the fact that at large input values the result is a very large number times a very small number. I am not sure if catastrophic ...
410 views

### What is the smallest positive integer that is not exactly representable as a floating point number in this system?

I have difficulties answering Exercise 3.10 from the book Overton M. Numerical computing with IEEE floating point arithmetic, 2nd edition. What is the smallest positive integer that is not exactly ...
21 views

I need a numerically stable way to compute the following ratio: $$\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$$ All the parameters are real positive numbers, with $0 < a,b,c$ and $0 < x &... 2answers 154 views ### Choosing epsilons Most numerical algorithms require an epsilon to be chosen in order to be robust and provide meaningful results. Choosing machine epsilon is usually too aggressive. Barring any special knowledge ... 1answer 136 views ### Compute hypergeometric function ratio:$\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$? I need a numerically stable way to compute the following ratio: $$\frac{_{2}F_{1}(a+1,b;c;x)}{_{2}F_{1}(a,b;c;x)}$$ All the parameters are real numbers, with$a< 0$,$\ b,c > 0$and$0<x&...
45 views

After studying the process of Gradual Underflow, I'm left a little curious as to why machines don't implement Gradual Overflow; where numbers exceeding the overflow level would be stored as un-...
309 views

### Overcoming floating point issues when Inverse does not exist but determinant provides nonzero result

I have an expression (let's say determinant of matrix A) expressed in symbolic form in terms of 2 decision variables x1, x2 and 2 parameters q1 and q2. I'm minimizing this using fmincon for different ...
147 views

### Accurate Polynomial Evaluation in Floating Point

What are the most accurate algorithms for evaluating a polynomial using floating point arithmetic? The internet seems to suggest that Horner's method is commonly used. In particular I have a cubic ...
21k views

### How to determine the amount of FLOPs my computer is capable of

I would like to determine the theoretical number of FLOPs (Floating Point Operations) that my computer can do. Can someone please help me with this. (I would like to compare my computer to some ...
3k views

### Which algorithm is more accurate for computing the sum of a sorted array of numbers?

Given is an increasing finite sequence of positive numbers $z_{1} ,z_{2},.....z_{n}$. Which of the following two algorithms is better for computing the sum of the numbers? ...
91 views

### Re-scaling array of floats so that all items are approximately integer

I have an array of floating point values $F$. I want to input my array into an algorithm that only takes integer values. How can I efficiently determine the smallest multiplier $m$ such that all ...
178 views

### Machine epsilon does not limit relative rounding error for denormals. Is this a problem?

As we know, machine epsilon limits relative rounding error in the range of normalized floating point numbers. But it is easy to check that this is not true for denormalized numbers. My question is ...
1k views

### range of positive mantissa in given floating-point number representation

I am a student and I came to this question while solving problems regarding the float-points. ...
4k views

### Machine epsilon (eps)

The wiki for machine epsilon says: "Machine epsilon gives an upper bound on the relative error due to rounding in floating point arithmetic" If machine epsilon is the upper bound on the relative ...
753 views

### How to handle floating point operations in HLSL?

I'm trying to write a perona malik anisotropic diffusion filter for the GPU. I'm basing my shader off a matlab implementation of the filter. I'm running into trouble because of what I suspect is ...