31 votes
Accepted

Why do we usually not want the eigenvalues of non-symmetric matrices?

Stability under perturbations Let $E$ be a perturbation such that $\|E\| \leq \varepsilon$. If $A$ is symmetric, then the eigenvalues of $A+E$ are at a distance $\varepsilon$ from those of $A$. (Bauer-...
28 votes
Accepted

Is half precision supported by modern architecture?

Intel support for IEEE float16 storage format Intel supports IEEE half as a storage type in processors since Ivy Bridge (2013). Storage type means you can get a memory/cache capacity/bandwidth ...
  • 2,096
27 votes
Accepted

Conserving Energy in Physics Simulation with imperfect Numerical Solver

There are a few ways to conserve energy during ODE integration. Method 1: Symplectic Integration The cheapest way that is to use a symplectic integrator. A symplectic integrator solves the ODE on a ...
23 votes
Accepted

Scientific computing vs numerical analysis

Wikipedia gives a good definition Numerical analysis is the study of algorithms that use numerical approximation (as opposed to general symbolic manipulations) for the problems of mathematical ...
20 votes
Accepted

Use of machine learning in computational fluid dynamics

It's a long-running joke that CFD stands for "colorful fluid dynamics". Nevertheless, it is used -- and useful -- in a wide range of applications. I believe your discontent stems from not ...
17 votes
Accepted

Why is RK45 used as the "default" method for non-stiff ODEs rather than a multistep one?

First, let's establish that they are a good choice. The SciMLBenchmarks are probably the most comprehensive that there are as of right now for modern methods. This uses the vast number of methods ...
17 votes
Accepted

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

On a basic level, I don't buy the argument that you have to "solve a linear system for many machine learning algorithms". Much more, you usually have to optimize a non-linear equation which ...
  • 2,937
14 votes
Accepted

Compute powers close to zero

Use the Taylor series expansion of $a^x$ about $x=0$ and evaluate a small number of terms for $a=10$: $$a^x - 1 \approx x\log(a) + \frac{1}{2}x^2\log^2(a)+\cdots.$$
13 votes

Compute powers close to zero

Personally, in the absence of a special function like expm1 (see also this scicomp.SE question), I'd use a Padé approximant instead of a Taylor/Maclaurin series; ...
  • 3,115
13 votes
Accepted

Why is the central difference method dispersing my solution?

I'll write the equation short as $$\ddot x(t)+c\dot x(t)=a(t,x(t))$$ to separate the "easy" linear parts from the non-linear and forcing terms. On the first method The claimed order of the ...
  • 4,829
13 votes
Accepted

How can I numerically integrate the Kepler problem?

I'm going to assume for the moment that your code is correctly implemented and that this problem isn't a bug. Believe it or not, gradual increase of the energy is the expected behavior of most simple ...
12 votes

Runge-Kutta in the presence of an attractor

The problem you are encountering is likely not a consequence of your choice of algorithm, but in fact a consequence of the resulting dynamical system after applying time reversal. Per the definition ...
  • 1,262
11 votes
Accepted

How important is learning hardware/architecture for scientific computing?

I haven't worked in quantum chemistry specifically, but I've worked in other areas where high performance is a correctness requirement (along with scientific accuracy), so I think we're on the same ...
  • 351
10 votes
Accepted

Analytical convergent sequence and numerical divergent sequence

Jean-Michel Muller, et. al., "Handbook of Floating-Point Arithmetic 2nd ed.", Birkhäuser 2018, gives the following example due to Muller, specifically constructed to deliver incorrect results with ...
  • 1,320
10 votes
Accepted

Going back in time in an initial value problem

This is technically still an IVP if you do an appropriate change of variables. Given your time is between $t \in [t^*, 0]$, make a new time variable $\tau = -t$ so that $\tau \in [0, -t^*]$ and you ...
  • 3,788
10 votes

What are the most important theorems in computational science?

You'll get everyone to give different answers to this question, and maybe that's alright. Here are some of my favorite ones: Taylor's theorem that a function (of sufficient smoothness) equals its ...
10 votes

How to solve a second order differential equation (diffusion) with boundary conditions using Python

I have found that I must keep the value of dt near dx or the results become unstable This behavior you have noticed is known as the Courant–Friedrichs–Lewy (CFL) condition. Indeed, there are ...
9 votes
Accepted

Is it possible to proof a-b+b = a for all double floating-point numbers?

You can sometimes prove such results (or get counterexamples) using an SMT solver such as Z3 that supports floating point arithmetic. Here is a proof of a version of your theorem that says $|((x+y)-y)-...
  • 11.4k
9 votes
Accepted

Poorly conditioned, easily evaluated sum for unit testing

The condition number of sum $s(x) = \sum_{j=1}^n x_j$ is given by $$ \kappa(x) = \frac{\sum_{j=1}^n |x_j|}{|\sum_{j=1}^n x_j|} = \frac{s(|x|)}{|s(x)|}$$ and reflects the sums sensitivity to small ...
9 votes
Accepted

Solve linear system with Newton-Raphson method

Yes you can do this, and it will converge in one iteration regardless of the starting value. This is because each step of Newton's method involves solving a linear system with the Jacobian of the ...
9 votes
Accepted

Accurate computation of Gauss-Kuzmin entropy

It's fairly easy to evaluate, to do this expand the logs in Taylor series in $x=(k+1)^{-2}$: $$ \log_2(1-x) = \frac{-1}{\log 2}\sum_{m\geq1}\frac{x^{m}}{m}$$ $$ \log_2(-\log_2(1-x)) = \frac{\log x}{\...
  • 11.4k
9 votes

Why in scientific papers convergence of finite difference and finite volume schemes is tested using multiple norms ($l_1$, $l_2$ and $l_{\infty}$)?

In finite dimensional spaces (say, in $\mathbb R^n$), all norms are equivalent and as a consequence, if something converges with a specific rate in one norm, it also converges with the same rate in ...
8 votes

Scientific computing vs numerical analysis

As someone who moved from Engineering to Scientific Computing during Grad school as an incidental need of the kind of work I was doing here are my two cents: Numerical analysis would focus on the ...
8 votes

ODE $x''(t)+\eta x'(t)+x(t)=0$ with the $\eta$ extremely small

The problem you have is what is called "stiff": it has two time scales, namely one for the oscillation (which is ${\cal O}(1)$) and one for the damping (which is ${\cal O}(\eta^{-1})$). If $\eta$ is ...
8 votes
Accepted

What kind of a researcher am I?

Up until a couple of decades ago, science was based on two large pillars. Those were theory and actual physical experiments. It is an exciting time to see a third pillar arise with numerical ...
  • 2,411
8 votes

Is half precision supported by modern architecture?

In my opinion, not very uniformly. Low precision arithmetic seems to have gained some traction in machine learning, but there's varying definitions for what people mean by low precision. There's the ...
  • 4,604
8 votes

Is half precision supported by modern architecture?

The accepted answer provides an overview. I'll add a few more details about support in NVIDIA processors. The support I'm describing here is 16 bit, IEEE 754 compliant, floating point arithmetic ...
8 votes
Accepted

A priori FEM estimates without $H^2$ regularity

The usual argument for error estimates in the energy norm is to first use the best-approximation property to get things back to the interpolation error. That is, $$ \| u-u_h \|_{H^1} \le C \| u-u_I \...
7 votes

Use of machine learning in computational fluid dynamics

I think you are mixing a couple different ideas that are causing confusion. Yes, there are a wide variety of ways to discretize a given problem. Choosing an appropriate way may look like "voodoo" ...
  • 4,587
7 votes

ODE $x''(t)+\eta x'(t)+x(t)=0$ with the $\eta$ extremely small

Let us write the equation in state-space form \begin{equation} \frac{d}{dt}\begin{bmatrix}x_{1}(t)\\ x_{2}(t) \end{bmatrix}+\begin{bmatrix}\eta/2 & -\omega\\ \omega & \eta/2 \end{bmatrix}\...

Only top scored, non community-wiki answers of a minimum length are eligible