# Where can one obtain good data sets/test problems for testing algorithms/routines?

In evaluating the quality of a piece of software you are about to use (whether it's something you wrote or a canned package) in computational work, it is often a good idea to see how well it works on standard data sets or problems. Where might one obtain these tests for verifying computational routines?

• I intended this to be a Community Wiki post, and have thus flagged it for conversion. – J. M. Nov 30 '11 at 7:19
• isn't this question too broad, i.e. it depends on the algorithms / the nature of the problem this software is used to solve ? – Andre Holzner Dec 9 '11 at 21:53
• I really wanted this question to be community wiki, @Andre (as a "big list" of resources); I had flagged it for conversion, but I don't know why it wasn't converted. – J. M. Dec 10 '11 at 0:14
• @J.M. I've converted it. – David Ketcheson Jan 13 '12 at 9:12

If you are interested in conducting an analysis on sparse matrices, I would also consider Davis's University of Florida Sparse Matrix Collection and the Matrix Market.

The method of manufactured solutions is a standard for testing PDEs and other solvers. Most symbolic algebra systems have facilities for generating code, this is useful for creating manufactured solutions. SymPy and Maple have the function ccode, among others for this purpose.

A test set for IVPs (Initial Value Problems for ODE solvers) is currently maintained by people from the University of Bari, Italy, who took it over from CWI Amsterdam.

For testing graph partitioning algorithms, there is Walshaw's Graph Partitioning Archive.

In computational electromagnetism, there is a famous (or infamous because of the difficulties in some) set of test problems: Testing Electromagnetic Analysis Methods (T.E.A.M.).

Some of them really need seriously state-of-art numerical techniques to get the correct simulation results aligned with the experimental data. For example, the conductor-coil problem.

Another set of testing problems for Maxwell equations are compiled by Dauge: Benchmark computations for Maxwell equations for the approximation of highly singular solutions. The one in the famous (or infamous) Fichera cube:

any $\phi\in H^{1+\epsilon}$ and $\mathbf{E} = -\nabla \phi$ living on this cube will be a challenge to your numerical PDE codes.

Lastly numerical PDE, there is hpFEM's Benchmarks in 2D (Problems with Known Exact Solutions), I have been using the test problems in it for a long time to test my finite element codes. For example, the famous non-smooth near the origin of the L-shaped domain example $$\Delta u= 0,\quad \text{where }\;u = r^{\alpha}\sin(\alpha \theta).$$

If you are interested in benchmarking algorithms related to molecular structures, the pubchem database has a large collection of mostly organic molecules. This may be useful to compare predictions of molecular properties obtained with different models/programs. The site has several options for downloading large batches of molecules that satisfy some predefined criteria (e.g. chemical composition).

Arnold Neumaier maintains a stable of test problems for unconstrained and constrained optimization (nonlinear programming). Included in this collection are the now standard test problems for unconstrained optimization due to Moré, Garbow, and Hillstrom.

The CUTEr web site updates the CUTE test set mentioned on Arnold Neumaier's web site with some additional problems for optimization and linear solvers. In addition, it provides software tools for the testing and updating of linear algebra and optimization solvers.

Athena's tests if you're solving hyperbolic conservation laws.

We use weather data sets in our building energy simulation software. For the US, the data sets consist of weather observations taken (usually at airports) every hour for the preceeding 20 years.

For testing statistical algorithms, there is A Handbook of Small Data Sets by D.J. Hand, F. Daly, K. McConway, D. Lunn, and E. Ostrowski. Some of those data sets can be downloaded from here.

For testing multivariate statistical analyses and machine learning algorithms, there is the UCI dataset repository at http://www.ics.uci.edu/~mlearn/

Hans Mittelman's website is an excellent resource for navigating the current software options in numerical optimization. He includes his own benchmarks, as well as links to other benchmarks for test problems in optimization.

Alan Genz proposed a test suite of functions in the paper Testing multidimensional integration routines. I cannot find an online version of this paper, but references to it can be found in the papers about the CUBA library.

There is a collection of reference PDE-constrained optimization problems maintained by Roland Herzog at TU-Chemnitz here.

Good software must have been tested, and should say how the authors have tested and either provide the test data sets themselves (e.g. in the form of regression tests) or at least provide links to the data it was tested with.

If you're looking for large graphs or network data to test on. The Stanford Network Analysis Project (SNAP) has many large graph datasets typically in the form of an anonymized adjacency list. Some of their options include:

### Properties of Data

• Number of edges: anywhere from ~10 to ~400 million
• Number of nodes: anywhere from ~10 to ~100 million
• Edge types: directed, undirected, weighted, unweighted, signed, and unsigned.
• Network types: directed, undirected, bipartite, multigraph, temporal, labeled.

### Tools

• @J.M. no problem! I used some of their social-net datasets a while ago for a project and then stumbled across this stackexchange and thought it might be helpful here. – ryan Nov 10 '17 at 3:02

Data is easy; the API to get it can be tough. I recommend Quandl. This site has over 10 million publicly available data sets accessible through one, easy, REST-ful API. All data is returned in either CSV or JSON. Or, if programming isn't your strong suit, there are easy ways to get the data into Excel. R, Python and Ruby programmers will be right at home with native libraries.

• Welcome to Scicomp! I don't think this is the sort of data the question is about; to test algorithms, you not only need a data set, but also a corresponding known outcome (depending on the problem/algorithm) to compare your results to. – Christian Clason Oct 3 '14 at 8:22
• Thanks, @ChristianClason. I see what you mean. E.g. if software is for linear regression, the author is interested in the data sets as well as a set of vetted analysis results to test if the linear regression package is performing correctly. – Brian Risk Oct 3 '14 at 13:45