5
$\begingroup$

[I previously asked my question on StackOverflow but this site may be more appropriate]

In Matlab, I am currently using the MultiStart as an optimization algo in a parallel setup for a computer cluster. For example, this is my Matlab code:

opts = optimoptions(@fmincon,'Algorithm','sqp','Use Parallel','Always'); %The options for the algo. The key here is the Use Parallel 
problem = createOptimProblem('fmincon','objective',...
 @(x) x.^2 + 4*sin(5*x),'x0',3,'lb',-5,'ub',5,'options',opts); %x is the variable I wish to min/max
ms = MultiStart;
[x,f] = run(ms,problem,20) %20 stands for the number of initial random variables

This code works very well when I submit a batch job on the cluster for X processors. I am looking online for a similar algo that I could implement in parallel using Python and I am bit loss. Any similar algo is available for Python?

UPDATE As for my cluster requirements, multithreading is not acceptable. Usually I use mpi4py.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Have you looked at Coopr (which includes Pyomo)? $\endgroup$
    – Christian Clason
    Jul 2, 2014 at 17:14
  • $\begingroup$ Interesting ... not so trival to see how to implement a MultiSearch from their help guide but I will look into this more. Thanks $\endgroup$
    – Plug4
    Jul 2, 2014 at 19:18

1 Answer 1

Reset to default
6
$\begingroup$

From what I can tell, MultiStart is not an optimization algorithm per se, but a framework that runs a given optimization algorithm (in your code, SQP via fmincon) in parallel for a set of random starting points (i.e., a separate, independent, instance of fmincon for each starting point). Since the instances do not need to communicate at all, this does not require MPI (the technical term is "embarrassingly parallel"), so you can use any job server to launch the instances, each on a separate node.

Since Python is (if you so choose) much more low-level than Matlab, you can implement this yourself rather easily: Pick an implementation of a minimization algorithm (the closest thing to fmincon would be SciPy's scipy.optimize.minimize) and wrap it in a multiprocessing Pool; the module joblib seems to provide a convenient wrapper for this. If you run this on a cluster, you could try dispy.

On a higher level, Coopr (an optimization modeling framework similar to AMPL) provides capabilities for Solving Multiple Instances in Parallel.

Improve this answer
$\endgroup$
3
  • $\begingroup$ yes you are absolutely right with your first point about MultiSearch not being an optimizaiton algo per se. Putting parallel prog. aside, can scipy.optimize.minimize be set to start with random starting points? Also, correct me if I am wrong, but multiprocessing (I was told) will not work on cluster and I usually use mpi4py to parallel job on a cluster. Is Coopr my only alternative to run scipy.optimize.minimize parallel on a cluster? $\endgroup$
    – Plug4
    Jul 2, 2014 at 19:54
  • $\begingroup$ What I have in mind is if scipy.optimize.minimize can take multiple random starting values like in Matlab's Multisearch, then I augment the performance with mpi4py ... no idea if this is feasible. $\endgroup$
    – Plug4
    Jul 2, 2014 at 19:56
  • 2
    $\begingroup$ No, minimize takes only a single starting point. What you would do is write a loop, where in each iteration you select a random starting point and run a new instance of minimize on that and collect the results. Then you only need a framework that allows you to execute the loop in parallel. If multithreading is not acceptable, simply replace it with a more suitable job server for Python. The best idea is to ask your cluster admin what they support. $\endgroup$
    – Christian Clason
    Jul 2, 2014 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.