I've asked this question on stackoverflow 2 weeks ago, but, judging by zero response, that probably was the wrong forum. Therefore copying it here:
F0,...,Fn be functions with parameters
G be a composite function. Say, for
I'd like to globally optimize the parameters
p0,...,pn with making as few function calls as possible. There are several ways to optimize:
1) Ignore that
G is a composition of
Fis, treat it as a blackbox with parameters
p0,...,pn, and use any generic global optimizer.
2) Optimize hierarchically; for clarity let's assume
Loop A: Freeze parameters
p0,p1,p2 and optimize
p3 in isolation. because
G depends only on
p3 and the output of prior functions that have frozen params I only have to evaluate
F3; the outputs of other
Fs don't change between iterations.
Loop B: Freeze parameters
p2, and repeat the above loop A. Repeat for optimizing
Loop C: Freeze
p1, repeat loop B, thus optimizing
Loop D: Obvious.
This "dynamic programming" approach would lead to optimizing
G by, hopefully, making fewer iterations. I have a couple of questions though:
First, I'm not sure that such hierarchical optimization approach would lead to solution of the same quality as the generic "blackbox" approach. Has any research have been done regarding the quality of results produced by hierarchical versus blackbox optimization?
Second, are there any optimization packages available out there supporting hierarchical optimization like that? I'm aware of a few available generic blackbox optimizers (NLOpt, etc), but not generic hierarchical optimizers.