What is the most appropriate derivative free optimization algorithm

We can use random optimization/ derivative free/ direct search to find the minimum of some black box function $f$.

If I have some 2D black box function, $f(x,y)$ - which I know to be convex - what is the best derivative-free method to use?

i.e. from a speed perspective is it best to use e.g. Nelder-Mead, Compass/Pattern search, MADS etc?

• It would be helpful if you could define "appropriate". I assume you are looking for the highest-performance solution? How expensive is it to compute $f(x,y)$, say, relative to $exp(x)$? Is the computing platform highly parallel (e.g. a GPU)? – njuffa Mar 20 '18 at 19:24

• I got a glimpse of what Julia has to offer for numerical ODEs (thanks to some of your previous posts), and I must say I'm positively impressed. does Julia offer other functionality similar to that of Python's numpy/scipy? if so, could you point me to some good reference material that can get me started with the language? – GoHokies Mar 27 '18 at 18:51
• Python's numpy equivalent is just Julia's base library. See how it's all covered in the manual. For what SciPy offers, Julia offers the same stuff but through a confederation of packages (much like R). A good source to search through packages is JuliaObserver, for example you can find Optim.jl and JuMP near the top for optimization. The language is still relatively new (with 1.0 coming this summer) but has a lot of unique packages already. – Chris Rackauckas Mar 27 '18 at 20:01