I have only built one working numeric solution to 2d Navier-Stokes, for a course. It was a solution for lid-driven cavity flow. The course, however, discussed a number of schemas for spatial discretizations and time discretizations. I have also taken more symbol-manipulation coursework applied to NS.
Some of the numeric approaches to handle conversion of the analytic/symbolic equation from PDE to finite difference include:
- Euler FTFS, FTCS, BTCS
- Midpoint Leapfrog
- offset grid (spatial diffusion allows information to spread)
To me, at the time, these seemed like "insert-name finds a scheme and it happens to work". Many of these were from before the time of "plentiful silicon". They are all approximations. In the limit they. in theory, lead to the PDE's.
While Direct Numerical Simulation (DNS) is fun, and Reynolds Averaged Navier-Stokes (RANS) is also fun, they are the two "endpoints" of the continuum between computationally tractable, and fully representing the phenomena. There are multiple families of approaches that live interior to these.
I have had CFD professors say, in lecture, that most CFD solvers make pretty pictures, but for the most part, those pictures do not represent reality and that it can be very tough, and take lots of work, to get a solver solution that does represent reality.
The sequence of development (as I understand it, not exhaustive) is:
start with the governing equations -> PDE's
determine your spatial and temporal discretization -> grid and FD rules
apply to the domain including initial conditions and boundary conditions
solve (lots of variations on matrix inversion)
perform gross reality checks, fit to known solutions, etc..
build some simpler physical models derived from analytic results
test them, analyze, and evaluate
iterate (jumping back to either step 6, 3, or 2)
I have recently been working with CART models, oblique trees, random forests, and gradient boosted trees. They follow more mathematically derived rules, and the math drives the shape of the tree. They work to make discretized forms well.
Although these human-created numeric approaches work somewhat, there is extensive "voodoo" needed to connect their results to the physical phenomena they are meant to model. Often the simulation does not substantially replace real-world testing and verification. It is easy to use the wrong parameter, or not account for variation in geometry or application parameters experienced in the real world.
- Has there been any approach to let the nature of the problem define
the appropriate discretization, spatial and temporal differencing scheme, initial conditions, or solution?
- Can a high definition solution coupled with the techniques of machine learning be used to make a differencing scheme that has much larger step sizes but retains convergence, accuracy, and such?
- All of these schemes are accessibly "humanly tractable to derive" - they have a handful of elements. Is there a differencing scheme with thousands of elements that does a better job? How is it derived?
Note: I will follow up with the empirically intialized and empirically derived (as opposed to analytically) in a separate question.
Use of deep learning to accelerate lattice Boltzmann flows. Gave ~9x speedup for their particular case
Hennigh, O. (in press). Lat-Net: Compressed Lattice Boltzmann Flow Simulations using Deep Neural Networks. Retrieved from: https://arxiv.org/pdf/1705.09036.pdf
Repo with code (I think):
About 2 orders of magnitude faster than GPU, 4 orders of magnitude, or ~O(10,000x) faster than CPU, and same hardware.
Guo, X., Li, W. & Ioiro, F. Convolutional Neural Networks for Steady Flow Approximation. Retrieved from: https://autodeskresearch.com/publications/convolutional-neural-networks-steady-flow-approximation
Others who have looked into the topic about 20 years ago:
Muller, S., Milano, M. & Koumoutsakos P. Application of machine learning algorithms to flow modeling and optimization. Center for Turbulence Research Annual Research Briefs 1999 Retrieved from: https://web.stanford.edu/group/ctr/ResBriefs99/petros.pdf
This characterises the use of non-gradient methods in deep learning, an arena which has been exclusively gradient based. While the direct implication of activity is in deep learning, it also suggests that GA can be used as an equivalent in solving a very hard, very deep, very complex problem at the level consistent with or superior to gradient descent based methods.
Within the scope of this question, it might suggest that a larger-scale, machine-learning based attack might allow "templates" in time and space that substantially accelerate convergence of gradient-domain methods. The article goes as far as to say that sometimes going in the direction of gradient descent moves away from the solution. While in any problem with local optima or pathological trajectories (most high-value real-world problems have some of these) it is expected that the gradient isn't globally informative, it is still nice to have it quantified and validated empirically as it was in this paper and the ability to "jump the bound" without requiring "reduction of learning" as you get in momentum or under-relaxation.
It seems that google now has a contribution "how to find a better solver" piece of the AI puzzle. link This is a part of making the AI make the solver.
And now they are doing it, and doing it well...
Is that a 100x speedup for Navier-Stokes solution??
It could be argued that they could then deconstruct their NN to determine the actual discretization. I particularly like figure 4.
Update 2020 link
Update 2022 - Physics Informed neural networks allow state compression by a factor of 85x with only slight loss. (link) One of the values of being able to stop, store the state, move it to another computer, and be able to pick it up and restart at the stop-point and not the initial condition is that it can allow reproducibility. A decent near-estimate of a state means the compute engine can get to the actual with only a little work. A ML-based "picture" that is good enough can act just like all the compute overhead from the initial state to the good enough estimate.