1
$\begingroup$

I am currently working on solving a positive definite symmetric systems in parallel. The parallel direct solver I used is MUMPS. However, the performance and efficiency of the parallel direct solver are quite disappointing compared to the sequential direct solver (Spooles is the one I used). The number of unknowns are 100k and 200k respectively on the figure.

I am wondering that if the parallel direct solver always has this kind of efficiency or I might make some mistakes on setting up the MUMPS.

Moreover, I also consider using iterative solver from PETSc but I cannot find a good example of using the block jacobi preconditioner, so it will be very much appreciate if someone is able to provide an example for PCBJACOBI and PCASM (I heard that the direct solver can be used in the subdomains and combined by using this PCASM preconditioner).

enter image description here enter image description here

Improve this question
$\endgroup$
6
  • $\begingroup$ Are you using it on computing cluster with a distributed memory? $\endgroup$
    – Anton Menshov ♦
    Apr 3 '18 at 16:57
  • $\begingroup$ Also, your labels on the plot say 10k/20k, but the text has 100k/200k. Which one is a typo? $\endgroup$
    – Anton Menshov ♦
    Apr 3 '18 at 17:04
  • $\begingroup$ @Anton Menshov, yeah it is on cluster. The matrices are generated from finite element method, so the vertices are for the mesh; and 100k/200k unknowns for the linear system. $\endgroup$
    – shidi.yan1992
    Apr 3 '18 at 17:07
  • $\begingroup$ My bad. got it! $\endgroup$
    – Anton Menshov ♦
    Apr 3 '18 at 17:07
  • 3
    $\begingroup$ I think 100K or 200K unknowns is not a very large problem for a cluster, it should only take a few seconds even on just a workstation. You might just be measuring setup/teardown/symbolic analysis more than you're measuring parallel factorization. $\endgroup$
    – rchilton1980
    Apr 3 '18 at 17:13
5
$\begingroup$

Your problem is too small. You have to consider that to get good efficiency, each processor has to have enough work to offset the cost of communication. In other words, there is a threshold how many degrees of freedom there have to be below which efficiency deteriorates.

I don't know where that threshold is for MUMPS. But, to give just one example, for iterative solvers in the context of finite element methods, you typically need at least 50,000 to 100,000 unknowns per processor. So, if you only have 200,000 unknowns, then you will get reasonable scalability to maybe 4 processors, but probably not beyond. It is quite possible that that threshold is even large for direct solvers, but I do not really know.

If you really wanted to test this, I would suggest you look at a rather large problem -- say, 10 million unknowns -- and see how this scales. This will tell you something about strong scalability. Or, you take a sequence of problems -- say, of size 100,000 times the number of processors -- and measure how long it takes to solve these. This will tell you something about weak scalability.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.