I am reading a paper that summarizes a set of simulations. Basically, the authors are trying to minimize some function using different optimization algorithms. They conclude: "Our findings point to instances of convergence at points where the first- and second-order optimality conditions fail."

What does this mean in plain English -- that parameter updates have essentially stopped even though the algorithm has not identified an optimal point? My understanding is:

  • First-order conditions: Ensure that a critical point has been found -- maximum, minimum, or inflection.
  • Second order conditions: Determine that this point is of the nature we want (e.g. that it is indeed a minimum).

Is this correct? My knowledge of optimization is a bit rusty, so I just wanted to confirm that I understand.

  • 1
    $\begingroup$ If possible, could you link to the paper please, with a page number? $\endgroup$ – Kirill Feb 11 '18 at 19:46
  • $\begingroup$ An ungated link is here. Relevant discussion in sections VI (B-C) but the details are a bit over my head. The language I used is from Intro/Conclusion on p. 34, 57. $\endgroup$ – atkat12 Feb 11 '18 at 19:54
  • 1
    $\begingroup$ From a brief skim, I interpret this as "we tried a bunch of algorithms and found that they returned a result even in situations where the theory didn't guarantee it" -- which hardly comes as a surprise to anyone familiar with optimization methods... $\endgroup$ – Christian Clason Feb 11 '18 at 23:09
  • 1
    $\begingroup$ That sentence means almost certainly that $\nabla f \ne 0 $ and $\nabla\nabla^Tf$ is not positive definite, yet the algorithm exited. $\endgroup$ – Deathbreath Feb 12 '18 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.