Solving PDEs in parallel

Question

I have read different approaches on how to solve pdes in parallel which are discretized using finite element method. For example:

1. Non-overlapping domain decomposition approach as mentioned in https://imsc.uni-graz.at/haasegu/Papers/Douglas-Haase-Langer/textbook.pdf at chapter 5.2 . Each process works on each own domain and the solution vector has a consistent storage while right hand sides, residuals and stiffness matrix has an additive storage.

2. Distribute the mesh into N parts. Each processor has information about it's own sub-domain plus a ghost layer (Global solvers approach).

What are the differences between these two methods, pros and cons? Which parallelization method is used by FEM softwares?

Answer 1

Domain decomposition was developed in the late 1990s and early 2000s because it allowed the re-use of sequential PDE solvers: You only have to write a wrapper around it that sends the computed solution to other processors, receives other processors' solutions, and uses these as boundary values for the next iteration. This works reasonably well for the small numbers of processors that were used at the time (a few dozen to at most a few hundred), but the approach does not perform well with large numbers of processors.

The approach almost universally used today is the second method you outline, where we think of the mesh and the linear system as one global one; it just happens to be stored in a way that distributes the data to many processors. In other words, we don't decompose the problem into smaller problems, we just decompose the storage of the data associated with the one global problem. This has required a lot of software development in libraries such as PETSc, Trilinos, libMesh, or the deal.II project which I co-lead. But, on the upside, this perspective leads to methods that can be efficiently solved, and as a consequence, they have largely supplanted domain decomposition methods in the last fifteen or so years.