Simulation-based Optimization vs PDE-constrained Optimization

Question

What is the difference between Simulation-based Optimization and PDE-constrained Optimization?

Would studying a text on Simulation-based optimization be sufficient to understand and apply both?

Answer 1

Both approaches apply to the same problem (numerical minimization of functionals which involve the solution of a PDE, although both extend to a larger class of problems). The difficulty is that for all but academic examples, the numerical solution of the PDEs requires a huge number of degrees of freedom which a) means that it takes a long time and b) computing gradients and Hessians by finite differences is completely infeasible. There's two ways of dealing with this:

1. You can take the numerical solution of PDEs as a black box that spits out a solution given a specific choice of the design values. This allows you to evaluate the functional at a point, but not any derivatives. Luckily, there are a number of derivative-free optimization methods that (usually) work (somewhat) better than blind guessing.¹ This seems to be what you call simulation-based optimization.

2. You can use mathematical tools such as the implicit function theorem or Lagrange multiplier calculus to give an analytical, exact, characterization of derivatives of the solution of a PDE, which often turn out to be solutions to suitably linearized PDEs that can be solved numerically using the same (if not simpler) tools as for the original models. These can then be used to characterize and numerically calculate gradients and Hessians of the functional to be minimized, meaning you can apply faster methods (since they use more information) such as steepest descent, nonlinear conjugate gradients, or (quasi-Newton) methods. This is what PDE-constrained optimization does.

As you can see, the approaches are very different, and in particular require different mathematical backgrounds. It is thus not reasonable to expect that reading a book on simulation-based optimization will give you the tools to understand PDE-constrained optimization (nor will a book on PDE-constrained optimization tell you about derivative-free optimization methods). If you'd like to know more about the latter, a good introductory (albeit still mathematical) textbook is

De los Reyes, Juan Carlos, Numerical PDE-constrained optimization, SpringerBriefs in Optimization. Cham: Springer (ISBN 978-3-319-13394-2/pbk; 978-3-319-13395-9/ebook). x, 123 p. (2015). ZBL1312.65100.

There are also many excellent lecture notes to be found on the internet.

---

¹Yes, I'm biased. Of course, if all you have is a software that simulates some process and nobody will tell you what PDE exactly it (tries) to solve, simulation-based optimization is pretty much the only game in town.