# How should errors be reported in scientific libraries?

There are many philosophies in different software engineering disciplines about how libraries should cope with errors or other exceptional conditions. A few of the ones I've seen:

1. Return an error code with the result returned by a pointer argument. This is what PETSc does.
2. Return errors by a sentinel value. For example, malloc returns NULL if it couldn't allocate memory, sqrt will return NaN if you pass in a negative number, etc. This approach is used in many libc functions.
3. Throw exceptions. Used in deal.II, Trilinos, etc.
4. Return a variant type; for example a C++ function that returns an object of type Result if it runs correctly and uses a type Error to describe how it failed would return std::variant<Error, Result>.
5. Use assert and crash. Used in p4est and some parts of igraph.

Problems with each approach:

1. Checking for every error introduces lots of extra code. The values into which a result will be stored always have to be declared first, introducing lots of temporary variables that might only be used once. This approach explains what error occurred but it can be hard to determine why or, for a deep call stack, where.
2. The error case is easy to ignore. On top of that, many functions can't even have a meaningful sentinel value if the entire range of output types is a plausible result. Many of the same problems as #1.
3. Only possible in C++, Python, etc., not in C or Fortran. Can be mimicked in C using setjmp/longjmp sorcery or libunwind.
4. Only possible in C++, Rust, OCaml, etc., not in C or Fortran. Can be mimicked in C using macro sorcery.
5. Arguably the most informative. But if you adopt this approach for, say, a C library that you then write a Python wrapper for, a silly mistake like passing an out-of-bounds index to an array will crash the Python interpreter.

Much of the advice on the internet about error-handling is written from the point of view of operating systems, embedded development, or web applications. Crashes are unacceptable and you have to worry about security. Scientific applications don't have these problems to nearly the same extent, if at all.

Another consideration is what kinds of errors are recoverable or not. A malloc fail is not recoverable and, in any case, the OS out-of-memory killer will get to it before you do. An index out of bounds for an array size isn't recoverable either. For me as the user, the nicest thing a library can do is to crash with an informative error message. On the other hand, the failure of, say, an iterative linear solver to converge could be recovered from by using a direct factorization solver.

How should scientific libraries report errors and expect them to be handled? I realize of course that it depends on what language the library is implemented in. But as far as I can tell, for any sufficiently useful library, people will want to call it from some language other than the one it's implemented in.

As an aside, I think that approach #5 can be improved substantially for a C library if it defines a global assertion handler function pointer as part of the public API. The assertion handler would default to reporting file/line number and crashing. The C++ bindings for this library would define a new assertion handler that instead throws a C++ exception. Likewise, the Python bindings would define an assertion handler that uses the CPython API to throw a Python exception. But I don't know of any examples that take this approach.

• Another consideration is performance ramifications. How do these various methods affect the speed of the software? Should we use different error handling in "control" parts of the code (e.g. processing input files) versus the computationally expensive "engines"? – LedHead Sep 18 at 0:27
• Note that the best answer will differ by language. – chrylis -on strike- Sep 18 at 2:19

I'll give you my perspective, which is encoded in the deal.II project that you reference.

First, there are two kinds of error conditions: Errors that can be recovered from, and errors that can not be recovered from.

• The former is, for example, if an input file can't be read -- for example if you are reading information from a file such as \$HOME/.dealii that may or may not exist. The reading function should just return to the calling function for the latter to figure out what to do. It may also be that a resource isn't available at the moment but may be again in a minute (a remotely mounted file system).

• The latter is, for example, if you are trying to add a vector of size 10 to a vector of size 20: Try as you might, there is nothing that can be done about this -- there's a bug in the code that led to the point where we attempted to do the addition.

These two conditions should be treated differently, regardless of the programming language you're using:

• In the second case, since there is no recourse, terminate the program. You could do that by throwing an exception or returning an error code that indicates to the caller that nothing can be done, but you might as well abort the program right away since that makes it so much easier for the programmer to debug the issue.

• In the former case, an exceptional situation has arisen that could be handled. Even though C and Fortran had no means to express this, all reasonable languages that came later have incorporated ways into the language standard to deal with such "exceptional" returns by providing, well, "exceptions". Use these -- that's what they're there for; they're also designed in such a way that you can't forget to ignore them (if you do, the exception just propagates one level higher).

In other words, what I'm advocating for here (and what deal.II does) is a mixture of your strategies 3 and 5, depending on context. It's true that 3 doesn't work in languages like C or Fortran -- in which case one may argue that that is a good reason to just not use languages that make it difficult to express what you want to do.

I will note that some systems just shouldn't crash, even in cases where errors aren't recoverable. An example is where a set of functions is called repeatedly for a number of queries -- say, for evaluating a likelihood function for given inputs in a statistical sampling scheme. Maybe the evaluator can't deal with negative values as the problem doesn't make any sense in that situation (e.g., evaluating the stiffness of a metal plate of thickness $$x$$), but since the evaluator needs to be called repeatedly it shouldn't just crash but just throw an exception. In such cases, even though passing in a negative value is not recoverable, one should throw an exception rather than abort the program. I had disagreed with this stance a couple of years ago, but have changed my mind after the xSDK community software guidelines encoded the requirement that programs should never crash (or at least should have a way to switch from crash to exception -- so deal.II now has the option to make Assert throw an exception instead of calling abort().)

• I would just recommend the opposite: throw an exception when the situation cannot be handled and return an error code when it can be handled. The problem is that dealing with thrown exceptions is tricky: the application programmer must know the type of all possible exception to catch and handle them, otherwise the program will just crash. Crashing is ok and even welcome for situations that cannot be handled, because the crashing point is reported out-of-the-box with python, e.g., but for situations that can be handled, it is (mostly) not welcome. – cdalitz Sep 18 at 13:38
• @cdalitz: It's a design flaw of C++ that you can throw objects of any type. But any reasonable software (Trilinos excluded) only throws exceptions that are derived from std::exception, and these can be caught by reference without knowing the derived type. – Wolfgang Bangerth Sep 18 at 16:39
• But I strongly disagree with returning an error code for the reasons outlined in the original question: (i) Error codes are ignored far too often, and as a consequence errors are not handled at all; (ii) in many cases, there is simply no exceptional value that can reasonably be returned given that the function's return type is fixed; (iii) functions have different return types, and you'd have to define in each case separately what the "exceptional" value would be that represents an error. – Wolfgang Bangerth Sep 18 at 16:41
• WB wrote (sorry, the '@' trick does not work for some reason and the username is removed by StackExchage for some reason): "Error codes are ignored far too often". This holds even more for exception catching: not many software developers take the trouble of bracketing every function call in a try/catch block. But it is mostly a matter of taste: as long as the documentation clearly states whether and which exceptions a function throws, I can handle it. But again it could be said: the duty to write documentation is ignored far too often ;-) – cdalitz Sep 18 at 17:15
• But the point is that if you forget to catch an exception, then there are no downstream problems: The program just aborts. It's going to be easy to find where the problem happened. If you forget to check the error code, your program may crash at some later point because of an undefined internal state -- but where the original problem was remains entirely unclear. It's exceedingly hard to find these kinds of bugs. – Wolfgang Bangerth Sep 18 at 18:51