There are many philosophies in different software engineering disciplines about how libraries should cope with errors or other exceptional conditions. A few of the ones I've seen:
- Return an error code with the result returned by a pointer argument. This is what PETSc does.
- Return errors by a sentinel value.
For example, malloc returns NULL if it couldn't allocate memory,
sqrtwill return NaN if you pass in a negative number, etc. This approach is used in many libc functions.
- Throw exceptions. Used in deal.II, Trilinos, etc.
- Return a variant type; for example a C++ function that returns an object of type
Resultif it runs correctly and uses a type
Errorto describe how it failed would return
- Use assert and crash. Used in p4est and some parts of igraph.
Problems with each approach:
- Checking for every error introduces lots of extra code. The values into which a result will be stored always have to be declared first, introducing lots of temporary variables that might only be used once. This approach explains what error occurred but it can be hard to determine why or, for a deep call stack, where.
- The error case is easy to ignore. On top of that, many functions can't even have a meaningful sentinel value if the entire range of output types is a plausible result. Many of the same problems as #1.
- Only possible in C++, Python, etc., not in C or Fortran. Can be mimicked in C using setjmp/longjmp sorcery or libunwind.
- Only possible in C++, Rust, OCaml, etc., not in C or Fortran. Can be mimicked in C using macro sorcery.
- Arguably the most informative. But if you adopt this approach for, say, a C library that you then write a Python wrapper for, a silly mistake like passing an out-of-bounds index to an array will crash the Python interpreter.
Much of the advice on the internet about error-handling is written from the point of view of operating systems, embedded development, or web applications. Crashes are unacceptable and you have to worry about security. Scientific applications don't have these problems to nearly the same extent, if at all.
Another consideration is what kinds of errors are recoverable or not. A malloc fail is not recoverable and, in any case, the OS out-of-memory killer will get to it before you do. An index out of bounds for an array size isn't recoverable either. For me as the user, the nicest thing a library can do is to crash with an informative error message. On the other hand, the failure of, say, an iterative linear solver to converge could be recovered from by using a direct factorization solver.
How should scientific libraries report errors and expect them to be handled? I realize of course that it depends on what language the library is implemented in. But as far as I can tell, for any sufficiently useful library, people will want to call it from some language other than the one it's implemented in.
As an aside, I think that approach #5 can be improved substantially for a C library if it defines a global assertion handler function pointer as part of the public API. The assertion handler would default to reporting file/line number and crashing. The C++ bindings for this library would define a new assertion handler that instead throws a C++ exception. Likewise, the Python bindings would define an assertion handler that uses the CPython API to throw a Python exception. But I don't know of any examples that take this approach.