# Where can one obtain good data sets/test problems for testing algorithms/routines?

In evaluating the quality of a piece of software you are about to use (whether it's something you wrote or a canned package) in computational work, it is often a good idea to see how well it works on standard data sets or problems. Where might one obtain these tests for verifying computational routines?

-
I intended this to be a Community Wiki post, and have thus flagged it for conversion. – J. M. Nov 30 '11 at 7:19
isn't this question too broad, i.e. it depends on the algorithms / the nature of the problem this software is used to solve ? – Andre Holzner Dec 9 '11 at 21:53
I really wanted this question to be community wiki, @Andre (as a "big list" of resources); I had flagged it for conversion, but I don't know why it wasn't converted. – J. M. Dec 10 '11 at 0:14
@J.M. I've converted it. – David Ketcheson Jan 13 '12 at 9:12
Thanks @David! $\phantom{}$ – J. M. Jan 13 '12 at 9:26

If you are interested in conducting an analysis on sparse matrices, I would also consider Davis's University of Florida Sparse Matrix Collection and the Matrix Market.

-
There is also Matrix Market math.nist.gov/MatrixMarket – stali Mar 29 '12 at 0:52

The method of manufactured solutions is a standard for testing PDEs and other solvers. Most symbolic algebra systems have facilities for generating code, this is useful for creating manufactured solutions. SymPy and Maple have the function ccode, among others for this purpose.

-

A test set for IVPs (Initial Value Problems for ODE solvers) is currently maintained by people from the University of Bari, Italy, who took it over from CWI Amsterdam.

-
Some additional test sets for IVPs are given in this answer from J.M. on Math.StackExchange: math.stackexchange.com/a/59398 – David Ketcheson Dec 2 '11 at 13:25

For testing graph partitioning algorithms, there is Walshaw's Graph Partitioning Archive.

-

If you are interested in benchmarking algorithms related to molecular structures, the pubchem database has a large collection of mostly organic molecules. This may be useful to compare predictions of molecular properties obtained with different models/programs. The site has several options for downloading large batches of molecules that satisfy some predefined criteria (e.g. chemical composition).

-

The CUTEr web site updates the CUTE test set mentioned on Arnold Neumaier's web site with some additional problems for optimization and linear solvers. In addition, it provides software tools for the testing and updating of linear algebra and optimization solvers.

-

We use weather data sets in our building energy simulation software. For the US, the data sets consist of weather observations taken (usually at airports) every hour for the preceeding 20 years.

-

Arnold Neumaier maintains a stable of test problems for unconstrained and constrained optimization (nonlinear programming). Included in this collection are the now standard test problems for unconstrained optimization due to Moré, Garbow, and Hillstrom.

-

Athena's tests if you're solving hyperbolic conservation laws.

-

Hans Mittelman's website is an excellent resource for navigating the current software options in numerical optimization. He includes his own benchmarks, as well as links to other benchmarks for test problems in optimization.

-

For testing statistical algorithms, there is A Handbook of Small Data Sets by D.J. Hand, F. Daly, K. McConway, D. Lunn, and E. Ostrowski. Some of those data sets can be downloaded from here.

-

For testing multivariate statistical analyses and machine learning algorithms, there is the UCI dataset repository at http://www.ics.uci.edu/~mlearn/

-

Alan Genz proposed a test suite of functions in the paper Testing multidimensional integration routines. I cannot find an online version of this paper, but references to it can be found in the papers about the CUBA library.

-

Good software must have been tested, and should say how the authors have tested and either provide the test data sets themselves (e.g. in the form of regression tests) or at least provide links to the data it was tested with.

-