Python: Computation time Issue with mpi4py

Question

I am using in Python mpi4py to process in parallel 20 minimization functions. Each of the 20 worker processes the same algorithm but with different random initial starting values. What I don't understand is the time it takes to process my job. I expect each of the workers to take roughly the same amount of time to minimize the function but it varies from 300 seconds to 2000 seconds. Now I am not sure why this is the case. I am wondering if I am specifying my MPI job properly or this happens to be because it is meant to happen. Here's my code

from mpi4py import MPI
import os
import random
import nlopt

data = #load a dataset

#Set the range for each of the variables (parameters)
X1_ = arange(1.01,1.99,.01)
X2_ = arange(0.01, 0.9, 0.01)

comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()

# Draw random values for each of the parameters
X1 = random.choice(X1_)
X2 = random.choice(X2_)

#Set up the lower and upper bound for each variables:
X1min = 1.05
X1max = 1.99
X2min = .01
X2max = 0.9999999

def myfunc(x, grad):
    if grad.size > 0:
        grad = numgrad(myfunction, [x[0], x[1]],
                       step=1e-8) #numgrad is a function that computes the gradient but irrelevant with a derivative-free algo
    return myfunction([x[0], x[1]], data)

opt = nlopt.opt(nlopt.LN_NELDERMEAD, 2)
opt.set_lower_bounds([X1min, X2min])
opt.set_upper_bounds([X1max, X2max])
opt.set_min_objective(myfunc)
opt.set_xtol_rel(1e-8)
opt.maxeval = 10000
x = opt.optimize([X1, X2])
minf = opt.last_optimum_value()

Am I missing something in my MPI specification?

Answer 1

The code looks fine, at first glance -- it does what you say you want it to do.

There's virtually no communication in the code you posted, so one thing you could try to isolate the cause of the performance is to run nlopt on each of your 20 starting points in serial, one at a time. When you do this, make sure to set the seed of your random number generator to a fixed value, so that you always get the same random variates.

Timing them will either confirm or disprove what I suspect is the main cause: Nelder-Mead requires a different number of iterations to converge for different starting values, leading to the performance differences you observe.