Using multiple languages in scientific codes

Question

Short version: Is it ever a good idea to use multiple languages in scientific codes?

Long version: May be its just me but these days I often see scientific codes written in multiple languages. The argument is that one language is used for performance and other one for ease of use, pretty syntax, because everyone else is using it and what not.

But from a users point of view its twice the work. I mean life is difficult as it is and now you must learn not one but two languages if you wish to adapt or extend the code to suit your own needs (and not everyone is a star programmer who can be productive quickly). There is lot more complexity at every stage, e.g., added dependencies just to configure/build the code.

In my field I see that codes which are written in a single language tend to have many users which contribute to it over time. I am guessing it is also true for successful codes in other fields but I am not sure.

So my question is that is using multiple languages ever a good idea in traditional scientific computing or is it a case of over engineering.

The only situation I can imagine when it may be needed is when you have to couple two or more codes, i.e., when there is no easy way to pass/share information between them. But often in those cases it is better to just rewrite. The climate/weather modeling community does a lot of coupling work but most of their codes and even couplers are written in a single language.

If you do down vote then please do comment.

Answer 1

I think that the "two language approach" is sound and I feel very comfortable in using it.

When you start a new project from scratch you never know beforehand which will be the critical code sections of your implementation. It is important to be able to have a robust and easy to maintain prototypical code that can be validated and benchmarked. Once you have the prototype code working you can start optimising it.

The prototyping step is usually better accomplished in expressive and easy to code languages (e.g. python/numpy), while the second one requires rewriting in a low-level language small sections of your code. A common experience is that once you have optimised your bottlenecks there is actually no need rewrite the entire code, since speed-ups would be marginal.

A much quoted Knuth passage says

Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

So my point is that there is nothing wrong in writing that 3% in a different language.

As a final comment, let me add that it is always desirable to keep the slow "high level" code as a reference to what the fast "low level" code is supposed to do.

Answer 2

Most scientific computing is probably done with a more flexible higher level language and then dropping into a lower level language to deal with speed bottlenecks. For example python/cython/f2py, R/Rcpp, matlab/mex. The celebrated (or more cynically, hyped) language Julia is explicitly designed to avoid this multi-language workflow which they call the "two language problem", for example as explained here.

Answer 3

Yes, if you are comfortable with the consequences.

Engineering always involves tradeoffs. The usual justification for using multiple languages is that a lower-level language (Fortran, C, C++) is used for performance-critical sections of code, and a higher-level language is used to provide a convenient interface and rapid code development. Julia, MATLAB, and NumPy/SciPy are good examples of such projects.

This decomposition can also be convenient for users working with such a project; advanced functionality is more likely to be written in a compiled language, whereas basic functionality is more likely to be written in an interpreted language. Furthermore, there is nothing stopping a project from providing "slow" versions of the performance-critical code in the interpreted language.

You're right, though, that the tradeoff can be maintainability; if you want to modify or maintain the code yourself, you now have to know enough about two languages instead of one. Similarly, if you pick one language, you're stuck with the libraries, performance characteristics, and community of that language. Using two languages gives you slightly more flexibility when it comes to library selection, performance, etc.

As the number of languages used in a project increases, maintainability becomes more problematic, and strict modular decomposition is helpful. It's useful if the core code is written in one or two languages and there are interfaces to seven other languages. (e.g., PETSc is written in C, but C is close enough to C++ that it can be called from C++, and it has Fortran, MATLAB, Python, and Mathematica interfaces; I think a Julia interface is even in the works). It's not so helpful if the core code is written in many different languages because it exacerbates the maintainability problem you already identified.

Answer 4

I think it depends on what you are doing. I do typically mix languages. I'll describe my personal experience below, perhaps this'll help you decide for your application. While I'll be talking about my own way of working below, and my tool of choice is Mathematica, I believe the same advantages apply to other high level languages too.

---

I typically write some type of simulation in a low level (C++) language. It has some (numerical) input and produces some potentially complex numerical output. Originally I passed the input through command line parameters and input files, and wrote the output into text files.

Eventually it became necessary to run the program for many different input parameters, which is best automated. I used to use some mix or shell scripting and scripting languages (Python) do call my program with various parameters / input-files, and processed/visualized the output in Mathematica.

Eventually I figured out that it's much better to write a Mathematica interface for calling my C++ code instead of writing a command line interface and using input files. Now this is my standard workflow. The Mathematica interface turned out to be simpler and easier to write (once I learned how!), mainly because of how easy it is to transfer structured numerical data without the need to convert it to text first. Consider e.g. having to pass a list of 3D arrays. One would need to invent a special storage format to transfer these, or use a C++ library for some common data format such as HDF5, which Mathematica already has a high level interface for anyway.

Using a Mathematica interface also made a number of things very easy, such as:

• smarter mapping of the parameter space (e.g. applying various adaptive sampling algorithms, which are much easier to implement and test in a high level language);

• easily save the simulation state and resume later (because transferring structured numerical data between Mathematica and C is easy enough that I can afford to take the time to implement this)

• run various numerical algorithms with my simulation, such as multivariate optimization, root finding, etc. (because Mathematica already has many of these built in)

• Visualizing the results immediately and interactively, without having to go through several steps (write the data to a file, process it for visualization, read it into the plotting program). Suppose that a simple root finding method would fail, and it's not clear why. If I can play with the simulation interactively, and visualize results immediately, it's much easier to figure out what's going on.

• Easily running simulations in parallel using Mathematica's parallel tools, even on multiple computers (thanks to Mma's built-in communication protocol).

• If the output of the simulation is very large (hundreds of MB), I can't afford to store it all for all parameter values. I'd usually run some analysis on it (averages, variances, etc.), and store only the results. Using the high level language to drive the simulation allows me to run many more non-trivial types of analyses without having to implement them myself in C++.

The focus here is on the ease of implementing things using a high level language with lots of useful built-in functionality. All of this could be done differently too, but it would be much more work, so I likely wouldn't do it.

Now I typically write the Mathematica interface before writing a command line interface, and only do the latter if I actually need it (e.g. I have to share the code with people who don't use Mathematica).

This worked well for me because I already knew Mathematica well, and I already used it for processing and visualizing the output anyway. I am also familiar with Mathematica's convenient C interface, which does take some time to learn. Also, I usually write these simulations alone, which doesn't restrict the tools I can choose. If I were collaborating with someone on the code, I would choose to use tools that we both know well.

The same advantages apply when using any other similar high level language with a convenient C interface, such as Python/MATLAB/Julia/R/etc. I'm mentioning Mathematica here because that's what I am familiar with and I'm describing my own workflow.

---

I also regularly use MATLAB and R functions (and occasionally Java) from within Mathematica. I do this because convenient high level interfaces already exist between these languages. Otherwise I wouldn't bother. I believe Python is one of the most convenient languages to use in this manner, gluing together functionality from various packages.

Answer 5

My experience matches with k20's on this topic. When you start a project, you choose a language consistent with one set of performance / flexibility options. Of course, halfway through, maybe you need additional performance - and have to incorporate lower-level languages. If you are starting out with a well-developed plan, maybe you can avoid this.

This isn't exactly unusual even outside of scientific computation, either - performance-critical code is usually put in the fastest reasonable language (e.g. inline assembler).