Im currently implementing a Nelder-Mead algorithm in MATLAB that can deal with constraints. In detail, the constraints should be added to the objective function with a Lagrange Multiplier($\lambda$) and the new function is then minimized by Nelder-Mead. Looks something like this.

$$L(x_1, \ldots,x_n,\lambda_1, \ldots,\lambda_n) = f(x_1,\ldots,x_n)+ \lambda\left(c-g(x_1,\ldots,x_n)\right)$$

this works for constraints that are equations, $g(x_1,\dots,x_n)= c$

Questions:

1. How does it work with constraints that are upper and lower bounds? cant put them into the objective function like that

2. Are the multipliers to be treated as additional variables by the nelder mead or are they constants you need to set?

Thank you guys.

• Welcome to SciComp.SE! Can you give some more details on this constrained Nelder--Mead algorithm? In the classical Lagrange approach, you need to maximize with respect to the Lagrange multiplier (which is an additional unknown!); if you have inequality constraints, the multiplier has to satisfy additional complementarity constraints. I have a hard time seeing how this would fit into the simplex gradient framework you use to show convergence for Nelder--Mead. Aug 27 '16 at 22:05
• Hey! Thank you. Well for equality constraints, I just add the term as above to the objective function and turn it into an unconstrained optimization problem, which i then put into standard nelder mead algorithm. As I said I cant see how i can do this with inequality constraints. Aug 27 '16 at 22:20
• Perhaps you want to add penalty terms rather than Lagrange-multiplier terms? However that said, the Nelder-Mead algorithm is probably not recommended. For a robust approach with the same flavor, you may want to look at COBYLA. Aug 28 '16 at 16:45
• Hey GeoMatt22, thanks for the advice. Problem is this is a university project and I HAVE to use nelder mead and lagrange multiplier. I think im going to use Lagrange multiplier in case of just equality constraints and in other cases penalty. Aug 28 '16 at 18:20
• Could this work: Whenever a solution violates constraints, i give it a very high function value. Or will it just not converge in many cases? Aug 28 '16 at 18:23