I've asked this question on stackoverflow 2 weeks ago, but, judging by zero response, that probably was the wrong forum. Therefore copying it here:

Let F0,...,Fn be functions with parameters p0,...,pn and G be a composite function. Say, for n=3 G(x)=F3(p3,F2(p2,F1(p1,F0(p0,x)))).

I'd like to globally optimize the parameters p0,...,pn with making as few function calls as possible. There are several ways to optimize:

1) Ignore that G is a composition of Fis, treat it as a blackbox with parameters p0,...,pn, and use any generic global optimizer.

2) Optimize hierarchically; for clarity let's assume n=3:

Loop A: Freeze parameters p0,p1,p2 and optimize p3 in isolation. because G depends only on p3 and the output of prior functions that have frozen params I only have to evaluate F3; the outputs of other Fs don't change between iterations.

Loop B: Freeze parameters p0,p1, change p2, and repeat the above loop A. Repeat for optimizing p2.

Loop C: Freeze p0, change p1, repeat loop B, thus optimizing p1.

Loop D: Obvious.

This "dynamic programming" approach would lead to optimizing G by, hopefully, making fewer iterations. I have a couple of questions though:

First, I'm not sure that such hierarchical optimization approach would lead to solution of the same quality as the generic "blackbox" approach. Has any research have been done regarding the quality of results produced by hierarchical versus blackbox optimization?

Second, are there any optimization packages available out there supporting hierarchical optimization like that? I'm aware of a few available generic blackbox optimizers (NLOpt, etc), but not generic hierarchical optimizers.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.