13
$\begingroup$

Recently, I've been pushing my group to include more testing when writing their code. There were several major bugs that took a lot longer to catch than probably speaking was necessary, because we didn't have a good testing regime in place.

However, I suspect that having the appropriate tools to automate (or to help streamline) the process, would certainly be useful. On the other hand, I don't know the various options for C++ testing suites, and how to decide between them?

Are there guidelines for what to look for—and are there any that are specifically geared towards numerical applications?

Improve this question
$\endgroup$

4 Answers 4

Reset to default
11
$\begingroup$

The problem with testing numerical codes is that (i) you may not always know the exact output and you will only be able to save the result of a computation now to compare against later -- i.e., to do regression tests, and (ii) that results may differ by small amounts on different machines due to different round-off.

To see how deal.II does it, take a look here: http://www.dealii.org/developer/development/testsuite.html#regression_tests

Improve this answer
$\endgroup$
4
  • $\begingroup$ Good points about the limitations of unit testing. Regression testing is a good thing (certainly better than not testing at all because output is unknown; it can give warning signs about bugs). As for the machine round-off issue, does mitigating that devolve into choosing a good tolerance via trial and error? $\endgroup$
    – Geoff Oxberry
    Commented May 9, 2013 at 1:24
  • 2
    $\begingroup$ It's a constant pain. In more than 10 years of testing, we've never come up with a really good strategy to deal with it. Using numdiff instead of diff can help, but ultimately you need to designate one machine for which you store "0.3987" instead of the "0.3988" you get on another machine when the correct number is 0.39875. No matter where you set the threshold, you will always cut one number of another off at the wrong place. $\endgroup$
    – Wolfgang Bangerth
    Commented May 9, 2013 at 9:14
  • $\begingroup$ @WolfgangBangerth. There are certain compiler specific flags which make floating point behaviour more deterministic. E.g. /fp:strict|precise and /Qimf-arch-consistency:true (Intel compiler) or -fnounsafe-math-optimizations, -ffloat-store (GCC) can make your code more consistent and reproducible across platforms at the cost of performance. With some tweaking, this provides a special "reproducible" build, which can be used specifically for testing. $\endgroup$
    – André
    Commented Aug 7, 2017 at 11:05
  • $\begingroup$ @Andre -- oh yes, we tried all of these. It still is difficult :-) $\endgroup$
    – Wolfgang Bangerth
    Commented Aug 7, 2017 at 13:35
10
$\begingroup$

I've recently been using googletest for testing a couple numerical libraries that I work on, and have been very happy with it. You can write fairly simple tests very quickly or you can write complicated tests that require data initialization and so on. It also provides (like I'm sure many others do) ways to easily do floating point comparisons rather than bitwise.

Improve this answer
$\endgroup$
1
  • $\begingroup$ A nice thing about googletest is that they make it easy to include its source code in an application, so you don't have to make it a dependency. $\endgroup$
    – Geoff Oxberry
    Commented May 8, 2013 at 18:58
4
$\begingroup$

If you're building your code with CMake, then the ctest mechanism would be the obvious choice. It allows you to test your code manually via the command ctest, and also supports extensive nightly testing via CDash.

Improve this answer
$\endgroup$
1
$\begingroup$

For our computational biology C++ library (Chaste) we use http://cxxtest.com/. This is fairly simple to use, works well, it provides a few macros for testing with assert() style statements. For scientific computing these are generally simple direct comparisons with TS_ASSERT_EQUALS(a,b) or numerical comparisons with TS_ASSERT_DELTA(a,b,tolerance).

Extra macros can easily be written using these basic ones to compare your own vectors/matrices of choice too. Usefully, you can also check that your code throws appropriate warnings and error messages in given situations. You can browse some examples in the test folders of our source code here: https://chaste.cs.ox.ac.uk/trac/browser/trunk

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.