How should errors be reported in scientific libraries?

Question

There are many philosophies in different software engineering disciplines about how libraries should cope with errors or other exceptional conditions. A few of the ones I've seen:

1. Return an error code with the result returned by a pointer argument. This is what PETSc does.
2. Return errors by a sentinel value. For example, malloc returns NULL if it couldn't allocate memory, sqrt will return NaN if you pass in a negative number, etc. This approach is used in many libc functions.
3. Throw exceptions. Used in deal.II, Trilinos, etc.
4. Return a variant type; for example a C++ function that returns an object of type Result if it runs correctly and uses a type Error to describe how it failed would return std::variant<Error, Result>.
5. Use assert and crash. Used in p4est and some parts of igraph.

Problems with each approach:

1. Checking for every error introduces lots of extra code. The values into which a result will be stored always have to be declared first, introducing lots of temporary variables that might only be used once. This approach explains what error occurred but it can be hard to determine why or, for a deep call stack, where.
2. The error case is easy to ignore. On top of that, many functions can't even have a meaningful sentinel value if the entire range of output types is a plausible result. Many of the same problems as #1.
3. Only possible in C++, Python, etc., not in C or Fortran. Can be mimicked in C using setjmp/longjmp sorcery or libunwind.
4. Only possible in C++, Rust, OCaml, etc., not in C or Fortran. Can be mimicked in C using macro sorcery.
5. Arguably the most informative. But if you adopt this approach for, say, a C library that you then write a Python wrapper for, a silly mistake like passing an out-of-bounds index to an array will crash the Python interpreter.

Much of the advice on the internet about error-handling is written from the point of view of operating systems, embedded development, or web applications. Crashes are unacceptable and you have to worry about security. Scientific applications don't have these problems to nearly the same extent, if at all.

Another consideration is what kinds of errors are recoverable or not. A malloc fail is not recoverable and, in any case, the OS out-of-memory killer will get to it before you do. An index out of bounds for an array size isn't recoverable either. For me as the user, the nicest thing a library can do is to crash with an informative error message. On the other hand, the failure of, say, an iterative linear solver to converge could be recovered from by using a direct factorization solver.

How should scientific libraries report errors and expect them to be handled? I realize of course that it depends on what language the library is implemented in. But as far as I can tell, for any sufficiently useful library, people will want to call it from some language other than the one it's implemented in.

As an aside, I think that approach #5 can be improved substantially for a C library if it defines a global assertion handler function pointer as part of the public API. The assertion handler would default to reporting file/line number and crashing. The C++ bindings for this library would define a new assertion handler that instead throws a C++ exception. Likewise, the Python bindings would define an assertion handler that uses the CPython API to throw a Python exception. But I don't know of any examples that take this approach.

Answer 1

I'll give you my perspective, which is encoded in the deal.II project that you reference.

First, there are two kinds of error conditions: Errors that can be recovered from, and errors that can not be recovered from.

• The former is, for example, if an input file can't be read -- for example if you are reading information from a file such as $HOME/.dealii that may or may not exist. The reading function should just return to the calling function for the latter to figure out what to do. It may also be that a resource isn't available at the moment but may be again in a minute (a remotely mounted file system).

• The latter is, for example, if you are trying to add a vector of size 10 to a vector of size 20: Try as you might, there is nothing that can be done about this -- there's a bug in the code that led to the point where we attempted to do the addition.

These two conditions should be treated differently, regardless of the programming language you're using:

• In the second case, since there is no recourse, terminate the program. You could do that by throwing an exception or returning an error code that indicates to the caller that nothing can be done, but you might as well abort the program right away since that makes it so much easier for the programmer to debug the issue.

• In the former case, an exceptional situation has arisen that could be handled. Even though C and Fortran had no means to express this, all reasonable languages that came later have incorporated ways into the language standard to deal with such "exceptional" returns by providing, well, "exceptions". Use these -- that's what they're there for; they're also designed in such a way that you can't forget to ignore them (if you do, the exception just propagates one level higher).

In other words, what I'm advocating for here (and what deal.II does) is a mixture of your strategies 3 and 5, depending on context. It's true that 3 doesn't work in languages like C or Fortran -- in which case one may argue that that is a good reason to just not use languages that make it difficult to express what you want to do.

I will note that some systems just shouldn't crash, even in cases where errors aren't recoverable. An example is where a set of functions is called repeatedly for a number of queries -- say, for evaluating a likelihood function for given inputs in a statistical sampling scheme. Maybe the evaluator can't deal with negative values as the problem doesn't make any sense in that situation (e.g., evaluating the stiffness of a metal plate of thickness $x$), but since the evaluator needs to be called repeatedly it shouldn't just crash but just throw an exception. In such cases, even though passing in a negative value is not recoverable, one should throw an exception rather than abort the program. I had disagreed with this stance a couple of years ago, but have changed my mind after the xSDK community software guidelines encoded the requirement that programs should never crash (or at least should have a way to switch from crash to exception -- so deal.II now has the option to make Assert throw an exception instead of calling abort().)