I am working on improving the optimization process of some demographic modeling software so it can better fit demographic models to data. We'd like to decrease optimization time.
The time it takes to evaluate our objective function varies a lot, depending on the input values. The relationship between time to evaluate the objective function and the input is known. I am wondering if there are any optimization methods that consider the relative time cost of the objective function when choosing which points to evaluate.
As Paul requested, here are some salient features of this particular objective function:
- The number of parameters is moderate (~12ish)
- Our problem is non-convex, or at least there are narrow and flat "ridges" in the objective function surface. Right now we're dealing with this using multiple optimizations from different points, but we'd love to do better.
- The objective function is pretty smooth, although we can only calculate finite-difference approximations to derivatives.
- The evaluation cost is also a smooth function of the parameter values, and it's quite predictable. roughly speaking, for each parameter the cost to evaluate is high at one end of the range and low at the other end. So we have large regions of expensive-to-evaluate parameter sets, but we know where they are.