When one equality-constrained optimization is formulated, the method of Lagrange multiplier will be the choice for me. In Chapter 17 from the book Numerical Optimization, quadratic penalty method can be used for such case. However, it doesn't mention when one should select quadratic penalty method over method of Lagrange multiplier. I hope to know the advantage and disadvantage of the two methods and how one select the method for one optimization problem.

  • 2
    $\begingroup$ In practice, augmented Lagrangian methods that combine penalty functions and the method of multipliers often provide better convergence than either approach on its own. $\endgroup$ Oct 12, 2020 at 4:29
  • $\begingroup$ @BrianBorchers Thanks. Augmented Lagrangian methods reduce the possibility of ill conditioning by introducing explicit Lagrange mulplier estimates into the function to be minimized compared to quadratic penalty method.(from the book Numerical Optimization). I don't understand why Lagrange multiplier method has convergence problem. $\endgroup$ Oct 12, 2020 at 12:30

1 Answer 1


The quadratic penalty is just easy to implement if you already have a solver for unconstrained problems. It converts the problem with constraints into an unconstrained one. It doesn't get any simpler. The penalty formulation also doesn't care about details such as whether the constraints are differentiable or not.

On the other hand, implementing a formulation that includes Lagrange multipliers substantially complicates the software. It also requires differentiability of the constraint functions and, furthermore, that these derivatives are computable in practice. It does, however, have the advantage, that at least for linear constraints, you can guarantee that all iterates satisfy the constraints exactly.

In other words, it's all about the various trade-offs involved.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.