# Repeated 1d minimization with similar parameters (scipy)

I have a function f(x,k1,k2) and I am trying to minimize it over x for different values of (k1,k2) on a 2d grid like so

for i,k1 in enumerate(np.logspace(-3.3,-1,20)):
for j,k2 in enumerate(np.logspace(-3.3,-1,20)):
if j==0:
initial_guess = best_x_0
else:
initial_guess = best_x
res = minimize(f,initial_guess,args=(k1,k2),bounds=((0.001,1),),tol=0.01,method='L-BFGS-B')


For given (k1,k2) this is a 1d problem and the function is relatively well behaved with only 1 minimum. However, evaluating it is very costly, ranging from a few seconds to up to about 10 minutes depending on the parameters. Obviously, there must be a more efficient way of solving this than treating it as many independent minimization problems. If the points (k1,k2) are close enough, and if I already have the minimum for one point, the minimum for the points around it should not be very different.

I looked at what Scipy has to offer but I did not find anything ideal for this purpose.

The functions in scipy.optimize.minimize_scalar do not require an initial guess and so I dont know how to take advantage of the 2d grid structure.

I also ran into issues using the 'L-BFGS-B' method of scipy.optimize.minimize. For example, using k1=k2=0.000501187233627 and an initial guess of x=0.02 it converges to x=0.022114610909 in 8 function evaluations. But if I use the same initial guess with k1=k2= np.logspace(-3.3,-1,20)[0] = 0.000501187233627272527534957103, when clearly there should not be any difference it get stuck on about x=0.01999926424 performing useless evaluations such as x=0.019966155,0.01999995105,0.0199951834.

What is the best way to accomplish this?

• I have had to tackle a similar issue where a given function evaluation took a few seconds each time, when the solution was striving to be as close to real time as possible. My end strategy ended up being to evaluate 3 initial points (say one for the upper and lower bounds and one at the average of the two) and fitting a quadratic. Then I would use the analytical minimization of the quadratic fit to evaluate a new point. I would update the quadratic fit with the nearest two points that were already evaluated and do this as many times as necessary. – spektr Apr 28 '16 at 4:52