As far as I can tell, the two big generic US Department of Energy computational science software frameworks are PETSc and Trilinos. They seem similar at first glance, beyond differences in language (C versus C++). What are the main differences between the two frameworks, and what factors should influence choosing one over the other? (Ignore institutional bias and existing infrastructure.)
There are huge differences in culture, coding style, and capabilities. Probably the fundamental difference is Trilinos tries to provide an environment for solving FEM problems and PETSc provides an environment for solving sparse linear algebra problems.
Why is that significant?
- Trilinos will provide a large number of packages concerned with separate parts of the FEM solver. Sometimes these packages work together sometimes they don't. Even the base components are in its own package and advanced C++ tools
- PETSC provides a small amount of core routines that can be built upon, but leaves the FEM solvers to third party packages. Because of this, it is associated with a larger community than just FEM. For example, even the eigen solvers are third party which is arguably a major part of linear algebra.
- Bottom line, Trilinos focuses working well within its own packages and PETSc has interfaces that call out to many middleware packages (I've often heard it called "lighter-weight" because of this but I wouldn't make that claim)
IMHO, which you should use really depends on the problem. Please share more details for us to answer that question.
If you want to hear from the technical leads of various packages, taking questions from the point of view of a normal HPC user, Brock Palen and Jeff Squyres' RCE podcast is an excellent resource. They have episodes on PETSc and on Trilinos which are very clear.
aterrel is right in his descriptions - PETSc is a (comparitively) small, integrated, well-thought out package of general-purpose linear and some nonlinear solvers, which one could then use in a solver framework; Trilinos is a collection of packages, with integration which is evolving, mostly aimed at being that solver framework, and includes things like ODE solvers, meshing packages, etc.
Let me add to aterrel's good comment that Trilinos is really a big bag of (Sandia) stuff and Petsc is a more focused library. If you want to compare then you should compare PETSc's sparse solver support with Trilinos' ePetra/ML/etc sparse solver ecosystem, that do similar things. Also, PETSc supports structured grids and Sandia has historically explicitly been an unstructured (FEM) house so Trilinos has little or no support for structured grids AFAIK. And Trilinos has capabilities that PETSc does not touch like stochastic PDE support.
As someone who has spent several years working with both, my perspective is that both packages aren't actually all that different. True, they use different languages, but they use it in very similar ways (both are object oriented, Trilinos makes little other use of C++ beyond using classes). Both support practically everything you will ever want to do with linear algebra (either through sub-packages or things they download on the fly, which from the user perspective doesn't make any difference). Finally, both have a good number of more obscure and probably not very widely used sub-packages (e.g. PETSc's mesh interface 'sieve', automatic differentiation in Trilinos, etc).
To me, the appeal of Trilinos is two-fold: - The number of obscure sub-packages in Trilinos is vastly larger; if I ever need something in direction X, I'm going to find it in Trilinos and it'll work with the rest of my code. - Trilinos is far more conservative in their development strategy. PETSc renames things all the time and every release requires users to catch up with renamed functions, different libraries, etc.
PETSc can very easily be used with Fortran and the documentation/examples are fairly good.
Trilinos to me looked confusing (with all its sub-packages and naming scheme) and Fortran support was spotty (at least when I looked at it a few years back).
There are two ways to interact with Trilinos from Fortran: (1) pass raw data into a C++ wrapper that makes all the calls to Trilinos packages for your [such wrappers exist for several packages] or (2) use the new ForTrilinos interfaces that heavily exercise the object-oriented features of Fortran 2003. Most of the features ForTrilinos requires are available in most compilers. the compiler implementations of these features is buggy but rapidly improving. ForTrilinos currently builds with the IBM and NAG compilers. Current or near-future releases of the Cray, Intel, and Portland Group compilers all nominally support the requisite features modulo bug fixes. The upcoming GCC 4.7.0 release will have all the requisite features but one, so wide compiler support is not too far in the future.