Where can one obtain good data sets/test problems for testing algorithms/routines?

Question

In evaluating the quality of a piece of software you are about to use (whether it's something you wrote or a canned package) in computational work, it is often a good idea to see how well it works on standard data sets or problems. Where might one obtain these tests for verifying computational routines?

(One website/book per answer, please.)

Answer 1

If you are interested in conducting an analysis on sparse matrices, I would also consider Davis's University of Florida Sparse Matrix Collection and the Matrix Market.

Answer 2

The method of manufactured solutions is a standard for testing PDEs and other solvers. Most symbolic algebra systems have facilities for generating code, this is useful for creating manufactured solutions. SymPy and Maple have the function ccode, among others for this purpose.

Answer 3

A test set for IVPs (Initial Value Problems for ODE solvers) is currently maintained by people from the University of Bari, Italy, who took it over from CWI Amsterdam.

Answer 4

For testing graph partitioning algorithms, there is Walshaw's Graph Partitioning Archive.

Answer 5

If you are interested in benchmarking algorithms related to molecular structures, the pubchem database has a large collection of mostly organic molecules. This may be useful to compare predictions of molecular properties obtained with different models/programs. The site has several options for downloading large batches of molecules that satisfy some predefined criteria (e.g. chemical composition).

Answer 6

The CUTEr web site updates the CUTE test set mentioned on Arnold Neumaier's web site with some additional problems for optimization and linear solvers. In addition, it provides software tools for the testing and updating of linear algebra and optimization solvers.

Answer 7

We use weather data sets in our building energy simulation software. For the US, the data sets consist of weather observations taken (usually at airports) every hour for the preceeding 20 years.

Data sets available for download.
Manual to describe the file format.

Answer 8

Arnold Neumaier maintains a stable of test problems for unconstrained and constrained optimization (nonlinear programming). Included in this collection are the now standard test problems for unconstrained optimization due to Moré, Garbow, and Hillstrom.

Answer 9

Athena's tests if you're solving hyperbolic conservation laws.

Answer 10

Hans Mittelman's website is an excellent resource for navigating the current software options in numerical optimization. He includes his own benchmarks, as well as links to other benchmarks for test problems in optimization.

Answer 11

For testing statistical algorithms, there is A Handbook of Small Data Sets by D.J. Hand, F. Daly, K. McConway, D. Lunn, and E. Ostrowski. Some of those data sets can be downloaded from here.

Answer 12

For testing multivariate statistical analyses and machine learning algorithms, there is the UCI dataset repository at http://www.ics.uci.edu/~mlearn/

Answer 13

Alan Genz proposed a test suite of functions in the paper Testing multidimensional integration routines. I cannot find an online version of this paper, but references to it can be found in the papers about the CUBA library.

Answer 14

Good software must have been tested, and should say how the authors have tested and either provide the test data sets themselves (e.g. in the form of regression tests) or at least provide links to the data it was tested with.