# What kind of optimisation algorithm is suitable for a computationally expensive function?

I have a reference value $R$ and a modelled value $M$. $M$ is generated using a stochastic algorithm with parameters $a$ and $b$.

The objective is to tune $a$ and $b$ so that $M$ is as close as $R$ possible (heuristically). But each run of $M$ takes around 30 seconds. Any "good" algorithm will be jammed by this computation, seemingly. Even so, what is better (than others) algorithm of choice?

If we are to allow $\vert M-R\vert$ converges quickly initially (and accept it to slowly converge in later stages), what types of algorithm should I look for? My colleague suggests Particle Swarm Optimisation (PSO). Is that a good choice?

Note: We need "good" fit but not "best" fit. Our tolerance is fairly large.

Response surface models (a kind of surrogate model) are often used in situations like this. The idea is to sample values of the parameters $a$ and $b$, compute $R(a,b)$ at each point, and then build a regression model (typically quadratic or even higher order) of the function. You can then optimize over the the fitted model.
• You can even improve the fit of the regression model each time you evaluate $R$ at the values $a$ and $b$ corresponding to the optimum of the current regression model. Commented May 14, 2016 at 21:05
• I would like to use the algorithm in R (The R Project, not $R(a,b)$ in the above). Which package is good? Commented May 17, 2016 at 5:29
• Just for others' interest, DiceOptim and DiceKriging seem to be good packages to handle model building, confidence intervals and expected improvements. Commented Jun 1, 2016 at 4:01