When one equality-constrained optimization is formulated, the method of Lagrange multiplier will be the choice for me. In Chapter 17 from the book Numerical Optimization, quadratic penalty method can be used for such case. However, it doesn't mention when one should select quadratic penalty method over method of Lagrange multiplier. I hope to know the advantage and disadvantage of the two methods and how one select the method for one optimization problem.
2
-
2$\begingroup$ In practice, augmented Lagrangian methods that combine penalty functions and the method of multipliers often provide better convergence than either approach on its own. $\endgroup$– Brian BorchersOct 12, 2020 at 4:29
-
$\begingroup$ @BrianBorchers Thanks. Augmented Lagrangian methods reduce the possibility of ill conditioning by introducing explicit Lagrange mulplier estimates into the function to be minimized compared to quadratic penalty method.(from the book Numerical Optimization). I don't understand why Lagrange multiplier method has convergence problem. $\endgroup$– Jogging SongOct 12, 2020 at 12:30
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The quadratic penalty is just easy to implement if you already have a solver for unconstrained problems. It converts the problem with constraints into an unconstrained one. It doesn't get any simpler. The penalty formulation also doesn't care about details such as whether the constraints are differentiable or not.
On the other hand, implementing a formulation that includes Lagrange multipliers substantially complicates the software. It also requires differentiability of the constraint functions and, furthermore, that these derivatives are computable in practice. It does, however, have the advantage, that at least for linear constraints, you can guarantee that all iterates satisfy the constraints exactly.
In other words, it's all about the various trade-offs involved.
answered Oct 10, 2020 at 21:00