Testing numerical optimization methods: Rosenbrock vs. real test functions

Question

There seem to be two main kinds of test function for no-derivative optimizers:

• one-liners like the Rosenbrock function ff., with start points
• sets of real data points, with an interpolator

Is it possible to compare say 10d Rosenbrock with any real 10d problems ?
One could compare in various ways: describe the structure of local minima,
or run optimizers A B C on Rosenbrock and on some real problems;
but both of these seem difficult.

(Maybe theorists and experimenters are just two quite different cultures, so I'm asking for a chimera ?)

See also:

• scicomp.SE question: where-can-one-obtain-good-data-sets-test-problems-for-testing-algorithms-routine
• Hooker, "Testing Heuristics: We Have It All Wrong" is scathing: "the emphasis on competition ... tells us which algorithms are better but not why."

Answer 1

Simple functions like Rosenbrock's are used to debug and pre-test newly written algorithms: They are fast to implement and to execute, and a method that cannot solve the standard problems well is unlikely to work well on real life problems.

For a recent thorough comparison of derivative-free methods for expensive functions, see Derivative-free optimization: A review of algorithms and comparison of software implementations. L.M. Rios, N.V. Sahinidis - doi 10.1007/s10898-012-9951-y Journal of Global Optimization, 2012. (See also the accompanying webpage: http://archimedes.cheme.cmu.edu/?q=dfocomp)

Answer 2

The advantage of synthetic testcases like the Rosenbrock function is that there is existing literature to compare with, and there is a sense in the community how good methods behave on such testcases. If everyone used their own testcase it would be much harder to come to a consensus which methods work and which don't.