C++ vs Fortran for HPC

Question

In my computational science PhD program, we are working almost exclusively in C++ and Fortran. It seems like some professors prefer one over the other. I am wondering which one is 'better' or if one is better than the other in a certain circumstance.

Answer 1

As so often, the choice depends on (1) the problem you are trying to solve, (2) the skills you have, and (3) the people you work with (unless it's a solo project). I'll leave (3) aside for the moment because it depends on everyone's individual situation.

Problem dependence: Fortran excels at array processing. If your problem can be described in terms of simple data structures and in particular arrays, Fortran is well adapted. Fortran programmers end up using arrays even in non-obvious cases (e.g. for representing graphs). C++ is better suited for complex and highly dynamic data structures.

Skill dependence: it takes a lot more programming experience to write good C++ programs than to write good Fortran programs. If you start out with little programming experience and only have so much time to learn that aspect of your job, you probably get a better return on investment learning Fortran than learning C++. Assuming, of course, that your problem is suited to Fortran.

However, there's more to programming than just Fortran and C++. I'd recommend to anyone going into computational science to start with a dynamic high-level language such as Python. Always remember that your time is more valuable than CPU time!

Answer 2

I think that both C++ and Fortran are good enough and work well.

However I think that Fortran is better for numeric scientific computing, for algorithms that can be expressed using arrays and don't need other sophisticated data structures, so in fields like finite differences/elements, PDE solvers, electronic structure calculations. Fortran is a domain specific language. In particular I think that it is easier to write fast programs in Fortran than in C++, by a scientist (not necessarily a computer science expert).

C++ is a general purpose language, so one can express any algorithm in it, and it is most definitely better for algorithms that can't be expressed using arrays, from HPC field probably some graphs, mesh generators, symbolic manipulation and so on.

It is also possible to write array algorithms in C++, but in my experience, it requires much more computer science knowledge and in general more work (i.e. one needs to create or reuse classes for array manipulation, and handle memory management by hand or using some library like Teuchos from Trilinos). Non-experts tend to write pretty good Fortran programs, but horrible C++ programs (talking from my own experience).

Disclaimer: I personally like Fortran a lot and I prefer it over C++ for numeric computing. I have spent over 2 years of programming in C++ daily, and almost a year programming in modern Fortran daily (in finite elements area). I use Python and Cython a lot too.

Answer 3

My approach has been to use C++ for everything but computational kernels, which are usually best written in assembly; this buys you all of the performance of the traditional HPC approach but allows you to simplify the interface, e.g., by overloading computational kernels like SGEMM/DGEMM/CGEMM/ZGEMM into a single routine, say Gemm. Clearly the abstraction level can be raised much higher by avoiding raw pointers and switching to opaque classes, but it is a nice first step.

I find the largest downside of C++ to overwhelmingly be the increase in compilation time, but, in my experience, the savings in development time more than make up for it. Another downside is that vendor C++ compilers tend to have more bugs than vendor C and Fortran compilers. In the past year, I think I have run into nearly ten bugs in C++ compilers.

With all of that said, I think that the undoing of scientific packages written in low-level languages (and Fortran) is the reluctance to expose convenient interfaces for sophisticated data structures: most people are satisfied with the Fortran BLAS interface, as it only requires pointers and leading dimensions to describe matrices, but few people would argue that the usual 40-integer Fortran sparse-direct solver interface is anything close to convenient (cf. UHM, SuperLU, PETSc, and Trilinos).

In summary, I argue for using assembly for low-level computational kernels, but higher level languages for everything else, especially when operating on non-trivial data structures.

Note that this post resulted in this comparison of the performance of C and Fortran on the kernel $y := \alpha x + y$.

Answer 4

Three facts:

• F77-style n-dimensional arrays in C: No problem using CnD (a shameless plug, admittedly)

• F90's module system is poorly designed and hostile to build environments. (A module's name doesn't have to match its filename, e.g.)

• Fortran does not support refactoring well. Pulling some bit of functionality out of a function requires you to touch four places: Actual code, variable declarations, argument declarations, and argument list. C gets by with two places to touch. This compounds the effect of the failure to manage data well (described below): Since small-scale modularity is so painful, just about everybody writes gigantic subroutines.

One personal impression:

• Fortran does not work well for managing data. Try returning a pointer to a user-opaque data structure in F77 or F90. (transfer(), here we come)

Answer 5

From my 15 years of thinking about scientific software: If your code runs 25% faster because you write it in Fortran, but it takes you 4 times as long to write it (no STL, difficulty implementing complex data structures, etc), then Fortran only wins if you spend a significant fraction of your day twiddling thumbs and waiting for your computations to finish. Given that for almost all of us the most valuable thing is our own time, the conclusion is this: use the language that allows you to develop, debug and test your code the fastest, within reason ignoring that it may be slower than maybe possible if you wrote it in Fortran.

Answer 6

I'm also throwing my two cents in kind of late, but I've only just seen this thread and I feel that, for posterity, there are a few points that desperately need to be made.

Note in the following that I will talk about C and not C++. Why? Well, otherwise it's apples and oranges to compare a full-fledged dynamically typed object-oriented language with something as static as Fortran. Yes, some modern implementations of the latest Fortran standards can do more than just that, but very few people actually use them, and so when we speak of Fortran, we think simple, static, and imperative language. That's where C is too, so I'll replace C with C++ for the following.

First of all, any discussion of Fortran/C having better compilers is moot. Dedicated C/Fortran compilers are a thing of the past. Both gcc/gfortran and icc/ifc are just different front-ends to the same back-end, i.e. your program will be transformed into an abstract description by the front-end and then optimized and assembled by the back-end. If you write, semantically, the same code in Fortran or in C, the compiler will, in both cases, produce the same assembly which will run just as fast.

This now leads to my second point: why do we still see differences? The problem is that most comparisons are made by Fortran programmers trying something in C or vice-versa. Ever notice how most authors or poets prefer to write in their native languages? Would you want to write poetry in a language in which you don't feel completely confident or at home? Of course not... I myself consider C to be my "native" programming language. I did, however, also spend three years working in a group that used only Fortran, in which I have achieved a certain level of fluency. I would, however, never write anything on my own in Fortran since I'm more comfortable with C and, as a consequence, the resulting code will be better, whatever you define that as.

So the main difference is in the programmer, not the language. So there are no differences? Well, not quite. Here are a few examples:

• SIMD: Whether it's SSE, SSE3 or AltiVec, if you want to use them in Fortran, you better hope and pray that the compiler guesses exactly what you want and does it so. Good luck. In C you generally have intrinsic functions for each architecture, or, more recently, general SIMD vector types in gcc. Most Fortran compilers will only use SIMD instructions to unroll loops, but if you have a kernel which works on short vectors of data in a non-obvious way, the compiler will very probably not see it.

• Different hardware architectures: The whole CUDA architecture is built around kernels in C. Yes, the Portland Group now has a CUDA-capable fortran compiler too, but it's commercial, and most importantly, it's not from NVIDIA. Same goes for OpenCL, for which the best I could find is a recent project which only supports a few basic calls.

• Parallel programming: Yes, both MPI and OpenMP work just fine with both C and Fortran. However, if you want real control of your threads, i.e. if you have a fully dynamic shared-memory computation, you'll be out in the cold with Fortran. In C you have the standard pthreads which, while not warm and fuzzy, will still get you through the storm. In general, most computations that rely on access to the operating system, e.g. threads, processes, file system, etc... are better served with C. Oh, and don't try to do your own networking with Fortran.

• Ease of use: Fortran is closer to Matlab than C is. Once you've gotten over all the different keywords and how to declare variables, the rest of the code looks like Matlab, making it more accessible to users with limited programming experience.

• Interoperability: When you create a struct in C, the layout of the actual data is straight-forward and deterministic. In Fortran, if you use pointer arrays or structured data, the actual layout of the data is strongly compiler-dependent, not straight-forward, and usually completely undocumented. You can call C from Fortran and vice-versa, but don't start thinking it may be as easy to pass anything more than a static array from one to the other and back.

This is all somewhat geeky, low-level stuff, but this is High-Performance Computing we're talking about, right? If you're not interested in how to best exploit the underlying hardware paradigms, i.e. implementing and/or developing algorithms which are best for shared/distributed memory, threads, SIMD vectorisation, GPUs using SIMT, and so-on, then you're just doing math on a computer.

This has gotten much longer that anything I entended, so here's a summary -- a set of take home messages of sorts:

• You will write the best code you can in the language which you know best.
• There is no difference in the quality of code produced by two compilers which use the same back-end -- it is us who write bad code in one language or another.
• Despite feeling more low-level, Fortran is quite a high-level abstraction and won't let you access certain hardware/OS features directly, e.g. SIMD, threads, networking, etc...

Answer 7

The problem with C++ is that you have numerous chances to ruin performance, for instance by blindly using STL, exceptions, classes (virtual overhead plus alignment problems), operator overloading (redundant new/deletes) or templates (never-ending compilation and cryptic errors seem benign, but you can waste hours this way).

However, more you gain better access to general libraries and possibly grater visibility of your code (although this strongly depends on the field, and you still have pure C). And you can still compensate the Fortran's lack of flexibility by wrapping its code in a script language like R, Lush, Matlab/Scilab or even Python, Ruby or Lua.

Answer 8

Since I am new here, I was looking through old questions and found this one. Hopefully it's not taboo to answer old ones!

Since no one else has mentioned this, figured I would. Fortran 2003 is almost fully supported by most of the major compilers (intel, ibm, cray, NAG, PCG) even gcc with the (soon-to-be) newest release 4.7. Fortran 2003 (and 2008) is an object oriented language, albeit a bit more verbose than C++. One of things that I think is nice about Fortran is the fact that the standard committee see's scientific computing as it's primary audience (I thank Damian Rouson for pointing this out to me the other day).

I bring this all up not so that C++ programmers become Fortran programmers, but so that Fortran people know that they have more options now besides switching to C++ or emulating object oriented concepts in Fortran 90/95.

One caveat I will add is that there is a cost to being on the bleeding edge of what's implemented in the compilers. If you undertake a major project in Fortran 2003 right now you will stumble across bugs and continually need to update your compiler (especially if you use gcc), though this has gotten significantly better in the past few months!

Answer 9

Fortran is optimized for array/matrix computations and is a thorough pain to work with for any type of text parsing. C and C++ may not match up with Fortran in numerical computing (it's close),but I find it much easier to process text and organize data (i.e. custom data structures) with C/C++.

As others have mentioned, don't count out dynamic interpreted languages (Python et al). They may not offer the face-melting speed of Fortan up front, but they allow you to focus more on solving your computational problem than all the details of implementation. Often you can implement a solution in Python, and if the performance is unacceptable, do some profiling, identify the problem areas, and either optimize that code using Cython or re-implement the entire program in a compiled language. Once you have the problem-solving logic fleshed out, the rest is just implementation and, with a good understanding of computing fundamentals, should be straightforward to represent in any variety of programming languages.

Answer 10

I'm currently working at one of the national labs. Most of the folks around me are mechanical engineers. Chatting with some of the folks in the HPC groups, they're doing mostly Linux and mostly C++. The group I'm currently in does mostly desktop applications and we use Windows and in descending order: C#, FORTRAN, Python, VBA and VB (6, not .NET). Some of the simulation engines we use were written at other national labs in FORTRAN.

Answer 11

What Jack P. I think is trying to say is that you should mix and match. A good piece of software is carefully layered. Different layers may map more naturally, or efficiently, to different languages. You should choose the most appropriate language for each layer. You should also understand how languages can interoperate, which may affect what language you choose for what layer.

A better question is what examples of excellently designed software are out there that are worth studying to learn about how to design layered software.