What makes Fortran fast?

Question

Fortran has a special place in numerical programming. You can certainly make good and fast software in other languages, but Fortran keeps performing very well despite its age. Moreover, it's easier to make fast programs in Fortran. I've made fast programs in C++, but you have to be more careful about things like pointer aliasing. So, there has to be a reason for this, and a very technical one. Is it because the compiler can optimize more? I would really like to know technical details, so if I use another language I can take these things into consideration.

For example, I know -or so I think- that one thing is that the standard specifies that pointers are contiguous in memory always which means faster memory access. I believe you can do this in C++ by giving a flag to the compiler. In this way it helps to know what Fortran does good, so that if using another language we can imitate this.

Answer 1

Language designers face many choices. Ken Kennedy emphasized two: (1) better abstractions and (2) higher- or lower-level (less or more machine-like) code. While functional languages like Haskell and Scheme focus on the former, traditional scientific-computing languages like Fortran and C/C++ focused on the latter. Saying that one language is faster than another is usually quite misleading: each language has a problem domain for which it excels. Fortran fares better in the domain of array-based numerical codes than other languages for two basic reasons: its array model and its explicitness.

Array Model

Fortran programmers largely do array manipulations. For that, Fortran facilitates several compiler optimizations that are not available in other languages. The best example is vectorization: knowing the data layout enables the compiler to invoke assembly-level intrinsics over the array.

Language Explicitness

While it seems that a simpler language should compile "better" than a more complex one, that really isn't the case. When one writes in an assembly language, there isn't much a compiler can do: all it sees are very-fine-grained instructions. Fortran requires explicitness (thus, more work by the programmer) only in cases that yield real rewards for array-based computing. Fortran uses simple data types, basic control flow, and limited namespaces; by contrast, it does not tell the computer how to load registers (which might be necessary for real-time). Where Fortran is explicit, it enables things like complete type inference, which helps novices to get started. It also avoids one thing that often makes C slow: opaque pointers.

Fortran Can Be Slow

Fortran is not fast for every task: that's why not many people use it for building GUIs or even for highly unstructured scientific computing. Once you leave the world of arrays for graphs, decision trees, and other realms, this speed advantage quickly goes away. See the computer language benchmarks for some examples and numbers.

Answer 2

The design of Fortran allows the compiler to perform stronger optimizations in some cases, optimizations that are not generally available to C.

One famous example is the handling of aliasing. In Fortran, you can access a specific memory area only though the specific symbol associated with that memory area. This knowledge allows the compiler to employ smart tricks when it's time to cache: it knows if a value has potentially changed or not. Until F90, this was verified easily. When Fortran 90 introduced pointers, the assumption was no longer true: you could access the same memory area through two (or more) symbols. This is the reason why you have to specify target to arrays you want to address via pointers.

Another interesting fact is that many constructs allow the compiler to perform parallelization without user intervention. Such luxury is possible due to the relative "platform agnosticism" of Fortran as a language.

There are many other subtle tricks like this. In addition, remember that nobody uses Fortran today, except for numerical calculations, meaning that the core feature and selling point of Fortran compilers is the resulting code speed. As a result, vendors focused on this.

Nevertheless, you can produce performant code also with other languages. It may, however, require special care or human intervention. The general point, however, is that performance is not a problem until there's a problem, and man time is way more expensive than computer time. So coding practices should focus on saving human time, rather than computer time.

Answer 3

I don't think Fortran is that close to the metal (see other answer) but it tends to optimize very easily. Loops are simple, and the language readily supports vectorization extensions (okay when I used it in my first job we were targeting a wide range of vector big iron).

There is also the large factor of inertia. A lot of numeric code is in Fortran, so builders of high end servers and supercomputers make sure they write good optimizing Fortran compilers. The compilers are good (even on machines with a relative lack of high quality compilers) so the users continue to use the Fortran and even write new code in it. So the builders make sure their next generation have good compilers, etc...

Answer 4

Be careful of urban myths here. If two compilers generate the same assembly code, then the resulting programs will have the same performance.

For any given piece of logic, there is a program in assembly language that minimizes its execution time. That program doesn't care which compiler generated it.

That said, compiled languages exist to make life easier for the programmer. Part of the cost of this is they may tempt the user into using features that don't result in minimal execution time. The prime example of this is new in C++. (How slow could it be - it's only three characters?) It practically begs you to dynamically allocate memory, and pay no attention to the runtime cost. If that's what you want to do, that's terrific, but Fortran could be faster just because it didn't suck you into doing that.

But way beyond that, I've never seen a program that, as first written, didn't have major room for performance improvement of the kind that the compiler could never clean up for you. As one example, spending a large fraction of time calling exp and/or log repeatedly with the same argument. As another example, calling DGEMM to multiply matrices and finding out that a large fraction of time goes into calling LSAME just to decipher its input character arguments.

This is at the same time as people are saying Fortran is faster because of pointer aliasing or loop unrolling. That's like saying a bus made by Porsche would certainly be faster than a bus made by Chevrolet. There needs to be a little common sense.