I am using in Python mpi4py to process in parallel 20 minimization functions. Each of the 20 worker processes the same algorithm but with different random initial starting values. What I don't understand is the time it takes to process my job. I expect each of the workers to take roughly the same amount of time to minimize the function but it varies from 300 seconds to 2000 seconds. Now I am not sure why this is the case. I am wondering if I am specifying my MPI job properly or this happens to be because it is meant to happen. Here's my code

from mpi4py import MPI
import os
import random
import nlopt

data = #load a dataset

#Set the range for each of the variables (parameters)
X1_ = arange(1.01,1.99,.01)
X2_ = arange(0.01, 0.9, 0.01)

size = comm.Get_size()
rank = comm.Get_rank()

# Draw random values for each of the parameters
X1 = random.choice(X1_)
X2 = random.choice(X2_)

#Set up the lower and upper bound for each variables:
X1min = 1.05
X1max = 1.99
X2min = .01
X2max = 0.9999999

def myfunc(x, grad):
    if grad.size > 0:
        grad = numgrad(myfunction, [x[0], x[1]],
                       step=1e-8) #numgrad is a function that computes the gradient but irrelevant with a derivative-free algo
    return myfunction([x[0], x[1]], data)

opt = nlopt.opt(nlopt.LN_NELDERMEAD, 2)
opt.set_lower_bounds([X1min, X2min])
opt.set_upper_bounds([X1max, X2max])
opt.maxeval = 10000
x = opt.optimize([X1, X2])
minf = opt.last_optimum_value()

Am I missing something in my MPI specification?

  • 2
    $\begingroup$ Please reduce your code to a minimal test case that correctly reproduces the problem. That will make it much easier for someone to help you. Throwing a bunch of code at people and expecting them to identify the problem by reading it is unhelpful and can also be considered rude. $\endgroup$
    – Kirill
    Commented Aug 23, 2014 at 0:29
  • $\begingroup$ @Kirill agree! I removed a lot of "pointless" lines. Thanks $\endgroup$
    – Plug4
    Commented Aug 23, 2014 at 2:09

1 Answer 1


The code looks fine, at first glance -- it does what you say you want it to do.

There's virtually no communication in the code you posted, so one thing you could try to isolate the cause of the performance is to run nlopt on each of your 20 starting points in serial, one at a time. When you do this, make sure to set the seed of your random number generator to a fixed value, so that you always get the same random variates.

Timing them will either confirm or disprove what I suspect is the main cause: Nelder-Mead requires a different number of iterations to converge for different starting values, leading to the performance differences you observe.

  • $\begingroup$ I agree with you. The Nelder-Mead algo may be the "issue". I chose a derivative-free optimization because the gradiant of my function is not trival to specify. Now I am wondering if I should choose another algo... I get the same results in Matlab using fmincon but it is not as fast... that's what makes me mad. $\endgroup$
    – Plug4
    Commented Aug 23, 2014 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.