Take the 2-minute tour ×
Computational Science Stack Exchange is a question and answer site for scientists using computers to solve scientific problems. It's 100% free, no registration required.

In evaluating the quality of a piece of software you are about to use (whether it's something you wrote or a canned package) in computational work, it is often a good idea to see how well it works on standard data sets or problems. Where might one obtain these tests for verifying computational routines?

(One website/book per answer, please.)

share|improve this question
    
I intended this to be a Community Wiki post, and have thus flagged it for conversion. –  J. M. Nov 30 '11 at 7:19
3  
isn't this question too broad, i.e. it depends on the algorithms / the nature of the problem this software is used to solve ? –  Andre Holzner Dec 9 '11 at 21:53
    
I really wanted this question to be community wiki, @Andre (as a "big list" of resources); I had flagged it for conversion, but I don't know why it wasn't converted. –  J. M. Dec 10 '11 at 0:14
    
@J.M. I've converted it. –  David Ketcheson Jan 13 '12 at 9:12
    
Thanks @David! $\phantom{}$ –  J. M. Jan 13 '12 at 9:26

17 Answers 17

If you are interested in conducting an analysis on sparse matrices, I would also consider Davis's University of Florida Sparse Matrix Collection and the Matrix Market.

share|improve this answer
1  
There is also Matrix Market math.nist.gov/MatrixMarket –  stali Mar 29 '12 at 0:52

The method of manufactured solutions is a standard for testing PDEs and other solvers. Most symbolic algebra systems have facilities for generating code, this is useful for creating manufactured solutions. SymPy and Maple have the function ccode, among others for this purpose.

share|improve this answer

A test set for IVPs (Initial Value Problems for ODE solvers) is currently maintained by people from the University of Bari, Italy, who took it over from CWI Amsterdam.

share|improve this answer
1  
Some additional test sets for IVPs are given in this answer from J.M. on Math.StackExchange: math.stackexchange.com/a/59398 –  David Ketcheson Dec 2 '11 at 13:25

For testing graph partitioning algorithms, there is Walshaw's Graph Partitioning Archive.

share|improve this answer

If you are interested in benchmarking algorithms related to molecular structures, the pubchem database has a large collection of mostly organic molecules. This may be useful to compare predictions of molecular properties obtained with different models/programs. The site has several options for downloading large batches of molecules that satisfy some predefined criteria (e.g. chemical composition).

share|improve this answer

Arnold Neumaier maintains a stable of test problems for unconstrained and constrained optimization (nonlinear programming). Included in this collection are the now standard test problems for unconstrained optimization due to Moré, Garbow, and Hillstrom.

share|improve this answer

The CUTEr web site updates the CUTE test set mentioned on Arnold Neumaier's web site with some additional problems for optimization and linear solvers. In addition, it provides software tools for the testing and updating of linear algebra and optimization solvers.

share|improve this answer

Athena's tests if you're solving hyperbolic conservation laws.

share|improve this answer

We use weather data sets in our building energy simulation software. For the US, the data sets consist of weather observations taken (usually at airports) every hour for the preceeding 20 years.

Data sets available for download.
Manual to describe the file format.

share|improve this answer

In computational electromagnetism, there is a famous (or infamous because of the difficulties in some) set of test problems: Testing Electromagnetic Analysis Methods (T.E.A.M.).

Some of them really need seriously state-of-art numerical techniques to get the correct simulation results aligned with the experimental data. For example, the conductor-coil problem.

Another set of testing problems for Maxwell equations are compiled by Dauge: Benchmark computations for Maxwell equations for the approximation of highly singular solutions. The one in the famous (or infamous) Fichera cube:

fichera

any $\phi\in H^{1+\epsilon}$ and $\mathbf{E} = -\nabla \phi$ living on this cube will be a challenge to your numerical PDE codes.

Lastly numerical PDE, there is hpFEM's Benchmarks in 2D (Problems with Known Exact Solutions), I have been using the test problems in it for a long time to test my finite element codes. For example, the famous non-smooth near the origin of the L-shaped domain example $$ \Delta u= 0,\quad \text{where }\;u = r^{\alpha}\sin(\alpha \theta). $$

share|improve this answer

For testing statistical algorithms, there is A Handbook of Small Data Sets by D.J. Hand, F. Daly, K. McConway, D. Lunn, and E. Ostrowski. Some of those data sets can be downloaded from here.

share|improve this answer

For testing multivariate statistical analyses and machine learning algorithms, there is the UCI dataset repository at http://www.ics.uci.edu/~mlearn/

share|improve this answer

Hans Mittelman's website is an excellent resource for navigating the current software options in numerical optimization. He includes his own benchmarks, as well as links to other benchmarks for test problems in optimization.

share|improve this answer

Alan Genz proposed a test suite of functions in the paper Testing multidimensional integration routines. I cannot find an online version of this paper, but references to it can be found in the papers about the CUBA library.

share|improve this answer

Good software must have been tested, and should say how the authors have tested and either provide the test data sets themselves (e.g. in the form of regression tests) or at least provide links to the data it was tested with.

share|improve this answer
    
Hear hear; see Reproducibility ff. and Reproducible-research-and-IPython-notebooks . –  denis Aug 29 '13 at 15:22

There is a collection of reference PDE-constrained optimization problems maintained by Roland Herzog at TU-Chemnitz here.

share|improve this answer

Data is easy; the API to get it can be tough. I recommend Quandl. This site has over 10 million publicly available data sets accessible through one, easy, REST-ful API. All data is returned in either CSV or JSON. Or, if programming isn't your strong suit, there are easy ways to get the data into Excel. R, Python and Ruby programmers will be right at home with native libraries.

share|improve this answer
    
Welcome to Scicomp! I don't think this is the sort of data the question is about; to test algorithms, you not only need a data set, but also a corresponding known outcome (depending on the problem/algorithm) to compare your results to. –  Christian Clason Oct 3 at 8:22
    
Thanks, @ChristianClason. I see what you mean. E.g. if software is for linear regression, the author is interested in the data sets as well as a set of vetted analysis results to test if the linear regression package is performing correctly. –  Brian Risk Oct 3 at 13:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.