In general, do iterative solvers require more floating point operations than the direct solver counterparts? I have some scientific code (written in both PETSc and FEniCS) for solving a mixed FE poisson equation and the profiling summaries seem to always show the direct solvers (eg, mumps, superlu) utilizing less flops by at least one order. Moreover, it also seems that for problems with over 1 million degrees of freedom, the solver requires more time, thus yielding an even smaller FLOPS/s efficiency than if I were to employ iterative solvers like GMRES (with bjacobi preconditioner).

Is this expected? Or is there potentially something wrong with my implementation(s)? Thanks

  • 2
    $\begingroup$ It would depend on the preconditioner used for the iterative solver. In practice I have found MUMPS (via PETSc) to be faster for time dependent problems with 1e6 DOF or less, specially if the structure of your matrix is not changing over time (the symbolic factorization is only done once). $\endgroup$ – stali Mar 2 '15 at 20:00

How are you counting FLOPS? If you are using the hardware performance counters on Intel chips starting with Westmere and later, then you may significantly over count them. These counters count instructions "executed" not "retired", but an executed instruction on these chips may not have actually been executed to completion and may be retried several times if the data is not available from RAM yet. This is very likely to happen with iterative solvers which are more likely to be based on matrix-vector products and therefore much more likely to be memory bandwidth limited. These codes are likely to be spending a lot of time waiting on data to be available from memory and will reissue the floating-point instructions many times. If you count FLOPS with the hardware counters on SandyBridge for the STREAM Triad benchmark, you will over count FLOPS by a factor of 10x or so.

This doesn't begin to address your other questions, but it could be an explanation of your problem depending on whether or not you're trying to use the hardware counters.


It really depends a lot on the problem. In my experience, direct solvers are very performant for 2D problems with moderate (say, up to approximately a million or so) numbers of degrees of freedom.

If you want to deal with very large problems, you typically want to use iterative solvers. You need very good preconditioners to achieve good performance, though; the most common would be multigrid preconditioners, either of the geometric or algebraic variety. The Jacobi preconditioner simply won't give good results.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.