# Guide for finite-difference schemes for Hamilton-Jacobi-Bellman Equations

I need to solve a simple, low-dimensional Hamilton-Jacobi-Bellman equation.

Is there a simple guide for doing this numerically using finite-difference schemes? I found a few research articles discussing higher order methods or discussing theoretical properties of numerical schemes but nothing that really helped me, as a newcomer without academic aspiration, to implement an actual algorithm. I really don't care about runtime and am perfectly fine with something that takes a whole day for 5% relative accuracy.

When I tried a simple forward Euler method, I got bizzare oscillations even though I tried to evaluate derivatives upstream. What upstream is depends on the solution of the Hamiltonian, of course, which itself requires evaluation of the derivatives; hence, there is some circularity here and I guess I got it wrong. Alternatively, I might have messed up the boundary conditions.

As an example of what I need to solve:

$$u\colon [0,T]\times \mathbb{R}^d\to \mathbb{R}$$ $$u_t-H(u,\nabla u)=0$$

$$H(u,\nabla u)=\min_{\alpha_i\geq 0\\ a\cdot \alpha=1} b\cdot \alpha+(c-\alpha)\cdot \nabla u$$ $$u(0,\cdot)=0$$

I tried the following iteration over finite two-dimensional arrays $$(u_n, \nabla u_n)$$, $$0\leq n\leq N$$ (with $$\Delta t:= T/N$$):

1. (For each array index) solve the Hamiltonian minimization problem to get $$H(u_{n},\nabla u_{n})$$ and $$\alpha$$.

2. (For each array index) define $$u_{n+1}:=u_n+\Delta t H(u_{n},\nabla u_n)$$

3. (For each array index) define $$\nabla u_{n+1}$$ by differencing $$u_{n+1}$$ in upstream direction, as defined by $$\alpha$$

At the boundary, I used $$u_{x_ix_i}:=0$$, i.e., I whenever the upstream method required me to access a non-existent entry of $$u$$, I evaluated downstream instead.