Let's say I'm optimizing something. To pick an arbitrary example, let's say I'm choosing the shape of some part to maximize strength-to-weight ratio. So I get some FEM software, parametrize the shape, and run gradient ascent or whatever to find the optimum shape.

When I do this procedure, I would generally expect that the calculated optimal strength-to-weight ratio is an overestimate of the actual strength-to-weight ratio for that shape---especially if I'm running the FEM software at low accuracy. Basically, the optimization algorithm will seek out and exploit any inaccuracies in the software to create impossibly high performance metrics.

In the context of economics, this basic insight is famously known as Goodhart's Law. In the context of optimization, how do people refer to this? Is there a term for it? Is there a paper or textbook that I can cite that discusses it?

  • $\begingroup$ On a less tangential note: The situation is not as simple, since the discretized parameter also has less degrees of freedom, limiting your optimization potential. Also, the difference is that these effects are of a mathematical, not psychological/social, nature, and hence amenable to mathematical analysis (at least up to the point of the continuum model). In your case, the keywords are "a posteriori error estimates" for "optimal control of PDEs" (or "PDE-constrained optimization") (also a currently active research topic). $\endgroup$ – Christian Clason Sep 30 '15 at 13:49
  • $\begingroup$ Just read wiki on Goodhart's Law, seems sort of overfitting in the statistics / machine learning field. Probably robust optimisation also has some relation to it. $\endgroup$ – jf328 Sep 30 '15 at 14:27
  • $\begingroup$ @ChristianClason -- I know it's not exactly the same, but the idea is that we have an ultimate goal (actual performance) and a proxy for it (simulated performance), and while no proxy is perfect, when you optimize based on the proxy it becomes a worse-than-usual and over-optimistic proxy. The important factor is "any kind of error or deficiency in my simulation software"--anything from poor choice of mesh, to oversimplifying the geometry, to actual programming errors. This is a more basic issue than error estimates, and not particularly amenable to mathematical analysis I think... $\endgroup$ – Steve Byrnes Sep 30 '15 at 14:29
  • $\begingroup$ @SteveB - My point (by which I still stand) was that there's two different issues at work here: a) you only optimize what you optimize (i.e., the mathematical model), which is such a basic observation that I don't think it has a name, and b) any numerical solution to the optimization problem will necessarily be approximate, but that this approximation is amenable to error analysis (such as for the specific case of error due to FEM discretization -- which you yourself brought up as a major influence -- of PDE constraints) and not as one-sided as you expect. $\endgroup$ – Christian Clason Sep 30 '15 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.