User friendly scipy optimize wrapper package?

Question

I'm creating too much throw away code for interfacing with the scipy optimize package in a user friendly way. (See code below for example of interruptible optimization that keeps last optimization value after a KeyBoard interrupt)

def monitoring_callback(x):
    global callback_dict
    callback_dict['cached_results'] = x
    callback_dict['counter']  = callback_dict['counter']  +1

try:
    fit_results= optimize.minimize(get_mean_squared_error,
                                   my_fit_params,method="CG", 
                                   callback=monitoring_callback)
    my_fit_params = fit_results

except KeyboardInterrupt:
    my_fit_params = callback_dict['cached_results']

Is there an existing package that does this sort of thing? (Also, if it implemented graphics like here, http://louistiao.me/notes/visualizing-and-animating-optimization-algorithms-with-matplotlib/

that would be really useful.)

Answer 1

It is generally better to encapsulate such stateful calls to library functions within a new function. Then monitoring_callback can become a local nested function, and callback_dict will be a regular variable with function scope, visible only to the nested function.

def compute_fit_params(my_fit_params):
    callback_dict = {'counter': 0}

    def monitoring_callback(x, state):
        callback_dict['cached_results'] = x
        callback_dict['counter'] += 1
        return False

    try:
        return optimize.minimize(get_mean_squared_error,
                                 my_fit_params, method="CG",
                                 callback=monitoring_callback)
    except KeyboardInterrupt:
        return callback_dict['cached_results'] if "cached_results" in callback_dict else None

my_fit_params = compute_fit_params()

This avoids suspicious-looking global variables (that persist between multiple calls to the same function!).

Note also that catching KeyboardInterrupt is something of an anti-pattern: in particular, if you try to genuinely interrupt the computation, you'd have to press ctrl-c twice. The results of your program are also much less deterministic because they depend on when the user, sitting at the keyboard, interrupted the program. It is generally better to limit the number of iterations the optimization algorithm can use, or else implement simple timeouts, like this, to let minimize terminate more gracefully:

callback_dict["start_time"] = time.process_time()
def monitoring_callback(x, state):
    callback_dict['cached_results'] = x
    callback_dict['counter'] += 1
    return time.process_time() > 10 + callback_dict["start_time"]