# The real myth of GPU (specifically CUDA) really speed up FEM/CFD

Now I have been believing that FEM/CFD is supposed to be faster on a GPU unit - here I am using CUDA as solid example. However, I have not been able to find a convincing paper where the benchmark actually appear to me that 'Yes, this is true!'. Can I please be pointed to one? Or if not, is there any reasons why GPU unit can suck when compared to CPU for CFD/FEM? Could it have anything to do with sparse matrices structure? In terms of performance index like speed/degree of parallelism etc.

• Why are you not believing it anymore? There are many reasons why a GPU code could be underperforming; badly written parallel code or code which isn't localized enough for good parallelization, low occupancy, many device<->host io operations, etc. I have seen a single GTX 980 perform on par with 1000 CPUs for embarassingly parallel codes. Imagine the cost of 1000 CPUs. – nluigi Oct 4 '16 at 8:23
• For a FE problem with a large number of degrees-of-freedom, the computational cost of solving the sparse system of equations will dominate. In FE structural analysis, direct solvers are almost always used as opposed to iterative solvers. This page, developer.nvidia.com/cholmod, shows relatively large speedups gained by solving sparse equations on a GPU. – Bill Greene Oct 4 '16 at 17:00
• You can't just dump it to GPU and expect speed-ups. If you can morph your problem into what GPU is specialized for then you can benefit from it – percusse Oct 5 '16 at 13:53
• Could you define "convincing" in the context of your question? A quick search at Google Scholar shows multiple relevant papers, why are these papers considered unsuitable? As far as I know, GPU acceleration is offered in shipping products from (for example) ANSYS, MCS, LSTC. – njuffa Oct 9 '16 at 18:46
• Because the majority of papers does not mention transferring data back and forth. For sprase structure and FEM where, for assembly for example, while most of the papers try to migrate the computing work to GPU there and keep it there! For very sparse matrices this is not quite practical, which is why I was expecting something like my selected answer. – Quang Thinh Ha Oct 10 '16 at 1:57

Here's the deal with GPUs. On a GPU, every single core is slow. Really slow. However, you have thousands of cores. If you can effectively use the thousands of cores at a time, then your algorithm will run better on the GPU. If you cannot, then it will run much slower on the GPU.

Linear algebra is one domain where parallelism is really well established. Thus the best way to write for a GPU is to essentially have the GPU do all of the linear algebra: it essentially becomes a card to compute Ax=b and A*B much faster than the CPU (this fact is pretty easy to check, look at the numerous benchmarks or even just open up MATLAB and type in A*B for both matrices and GPU matrices). But there's a caveat: data transfer to GPUs is really slow. Also, memory allocation on GPUs is really slow. So while the linear algebra is fast, you have to deal with the fact that:

1. Serial performance is awful.
2. Allocating memory dynamically on the GPU destroys performance.
3. Transferring back and forth between the CPU and GPU is slow.

This puts constraints on your algorithm: you need to try to leave as much on the GPU as possible, transferring back and forth the minimum amount, while trying to avoid serial parts from running on the GPU. Likewise, the GPU can easily do the linear algebra 1000x faster than the CPU (which is usually the performance bottleneck) so in many cases you can effectively manage this dilemma and end up with large performance gains.

One interesting alternative are the Xeon Phi. These cards have much faster data transfer, much better serial performance, and can allocate memory much better. However, the tradeoff is it's less specialized to be a "dumb linear algebra solver" and you thus have to pay a heftier price, and in return its linear algebra performance is about half that of a GPU. However, this can be much easier to develop code for (OpenMP parallel codes will use it automatically, and you can use a Xeon Phi card as another node via MPI, so if you've already parallelized a code you can use the same code with the Phi) and, by allowing you to more effectively keep data on the acclerator or using the increased data transfer speeds, can be much faster than a GPU in real-world (S)PDE solving. Of course, it depends greatly on the implementation.

• This is exactly what I was looking for. Thanks - also do you have some documents - either books or papers - to better explain this? – Quang Thinh Ha Oct 10 '16 at 2:16
• not saying your answer is bad!! (sorry!!!) but I just need a more in-depth details :D ) – Quang Thinh Ha Oct 10 '16 at 2:16
• Some of this I don't know a reference for, but it's easy enough to check yourself. You can see directly from the specs that serial performance is bad, but if you want to test it, just test what happens when you run a CUDA code on one CUDA thread vs one CPU thread. You can easily do this with Julia. Play with it a few times and you'll see. Then write a code that allocates memory. I'm sure you'll see the same thing I did. You can use the same tutorial to test the time for the data transfer. – Chris Rackauckas Oct 11 '16 at 2:05
• To test that the Xeon Phi allocates faster and does faster data transfer, I tested Monte Carlo Euler-Maruyama solvers on SDEs, memory allocation tested by dynamically allocating the array for the output. Both do not so well with memory allocation, but the Xeon Phi does reasonable while the GPU essentially grinds to a hault (which is why it's never recommended). – Chris Rackauckas Oct 11 '16 at 2:08
• Oh, and Xeon Phis don't have to be an acceleration card. You can either write the full algorithm on the card (since it can run all of the commands and has its own OS), or you can actually just have your CPU be a Xeon Phi. – Chris Rackauckas Jun 5 '17 at 7:06

To extend Chris Rackauckas's exhaustive answer with a reference try to look pdf by Torres, Gonzalez-Escribano, Llanos. It is about the tuning of a gpu, that is an important aspect for performance.

As Bill Greene's comment remembers the most relevant part of the computation work is about solve linear system, but however the assembly part can take a discrete amount of time. In this side gpu is a good help because this part has got a naturally high parallelism. You can calculate every element independent and at the end built the matrix.

(For my master thesis in mathematics I developed a parallel version with CUDA of a CFD solver write in matlab)

It really depends on how to solve the FE system with massive parallelism.