# How to numerically minimize a functional?

How to numerically minimize a functional, for example,

$$J[y]=\int_{x_1}^{x_2}L(x,y(x),y'(x))dx$$

An equivalent problem is to solve the Euler equation for this functional as a differential equation. But is there a numerical method to directly minimize this functional without solving a differential equation?

• Welcome to Scicomp.SE! You omitted some crucial information (structure of $L$, function space to which $y$ belongs, whether there are constraints), but what you are looking for is called "gradient descent". Maybe math.unt.edu/~jwn/flag.pdf is a good introduction; you could also look at doi.org/10.1016/j.na.2008.11.065 and follow the cited literature. – Christian Clason Apr 15 '18 at 13:32
• One way to minimize this is to discretize your $y(x)$ in some way, say by representing it with a polynomial basis with unknown coefficients, and then solving the resulting (nonlinear) optimization problem you get. You can use any typical optimization scheme at that point. – spektr Apr 17 '18 at 12:45

Variational problems like this are special cases of optimal control problems, for which there is a huge literature on solution methods and also a good amount of available software. To express it as in the standard form used in optimal control, we may write $t$ rather than $x$ and consider "system dynamics" $$\dot{y} = u$$ where $u$ is a (fictitious) system input. We can then write the cost functional as $$J = \int_{t_1}^{t_2}L(t,y,u)~dt$$ which is to be minimised by choosing our system input, $u$. It's common in optimal control problems to have more complex problems such as inequality constraints on $y$ or $u$, so this is actually significantly more general than the original calculus of variations problem you posed.

The are many numerical solution methods, but the most commonly used in practice are "direct" methods that express the cost as a sum rather than an integral by some process of discretisation. The most common approaches are known as direct collocation (see e.g. https://doi.org/10.2514/3.20223) and multiple shooting (e.g. https://doi.org/10.1016/S1474-6670(17)61205-9). After this we get a nonlinear programming (NLP) problem, which is an optimisation problem that can be solved using standard solvers.

If you're interested in solving practical problems, I would recommend taking a look at the open source toolkit ACADO, which solves optimal control problems by multiple shooting, or CasADi, which is an automatic differentiation toolkit designed for dynamic optimisation that can interface with common NLP solvers such as IPOPT.