# Saddle point formulation and minima

Given two Hilbert spaces $X$ and $Y$ and $a \colon X \times X \to \mathbb{R}$ as well as $b \colon X \times M \to \mathbb{R}$, both being continuous bilinear forms with $a(\cdot, \cdot)$ being symmetric and coercive for all $v \in X$. $f \in X'$ and $g \in M'$.

I am worried about the proper connection between the minimum formulation and the saddle point formulation.

Find the minimum $u \in X(g) = \{ u \in X \mid b(u,q) = g(q) \, \forall q \in M \}$ of the quadratic functional $J(u) := \frac12 \, a(u,u) - l(u)$. With other words:

$$\min_{u \in X(g)} \, J(u).$$

Now the Lagrangian Functional $L \colon X \times M \to \mathbb{R}$ is introduced, given as

$$L(u, \lambda) := J(u) + (b(u,\lambda) - g(\lambda)).$$

Now, for all $\lambda$ we do the following:

$$X \times M \to X \times {\lambda} \cong X \to \mathbb{R}$$

with finding $u_\lambda \in X$ for all $\lambda$.

Then: We are trying to find out, which $\lambda \in M$ do fulfill the condition that $u_\lambda \in X$. This must be a solution of the minimising problem.

Then there is a conversion to a system of equations of the approach given above, namely:

Find $(u, \lambda) \in X \times M$ such that $$a(u,v) + b(v, \lambda) = l(v), \quad \forall v \in X \\ b(u,\mu) = g(\mu), \quad \forall \mu \in M$$

I cannot see how to transfer the given procedure to the equations given above.

My thoughts so far: As we are dealing with a Lagrangian, the best seems to be to take the derivatives, both of $u$ and $\lambda$. What puzzles me most is what happens to the factor $\frac12$ in the defintion of $a(\cdot, \cdot)$. Is this gone due to the symmetry? How do the $v$ and $\mu$ come into place when we take a look at the derivative of the Lagrangian? Especially: how do we know which argument of $a$ can be replaced by $v$?

• are you familiar with the first order KKT (optimality) conditions for a (local) minimum (see [chapter 12 of this book)? Any local minimum $(u^*,\lambda^*)$ will have to verify your two linear equations. The first equation in the two-equation system is the so-called adjoint equation $L_u (u^*,\lambda^*)(v) = 0$, $\forall v \in X$. The second equation (nothing other than your "primal model" constraint) comes from $L_\lambda (u^*,\lambda^*)(\mu) = 0$, $\forall \mu \in M$. – GoHokies Jan 16 '18 at 17:31
• and yes, the 1/2 factor vanishes because of the symmetry and linearity of the bilinear form $a$. – GoHokies Jan 16 '18 at 17:31
• @GoHokies No, I am not. But thank you, I will get that book tomorrow. Helpful to have a point where to start now! – mdot Jan 16 '18 at 17:49

To expand on @GoHokies comment: The answer to your all your questions is basically "because $a$ is bilinear and symmetric" (necessarily so; without symmetry this approach wouldn't work). Specifically, don't think of gradients but of directional derivatives: You have a saddle point if the derivative in every direction vanishes, i.e., if the Fréchet derivative (you should look this up; understanding how derivatives of functionals are defined is important) of $L$ at $u\in X$ applied to every direction $\nu\in X$ gives zero: $$L'(u,\lambda)\nu = 0 \qquad\text{for all } \nu\in X.\label{cc:1}\tag{1}$$ Now $l$ and $b$ are linear in $u$, hence $l'(u)\nu = l(\nu)$ and similarly for $b$. The bilinear form is quadratic in $u$ and so by the product rule, $$\frac{d}{du} a(u,u)\nu = \frac{d}{du} a_1(u,u)\nu + \frac{d}{du} a_2(u,u)\nu = a(\nu,u) + a(u,\nu) = 2 a(u,\nu),$$ where $a_1$ is $a$ with the first argument frozen (i.e., you only take the derivative with respect to the first argument) and similarly for $a_2$, and I have used that $a$ is bilinear in each argument separately. The last step is of course due to the symmetry of $a$ (which also means that it doesn't matter which argument of $a$ is replaced, answering your third question).
Hence, \eqref{cc:1} reduces to $$a(u,\nu) + b(\nu,\lambda) = l(\nu)\qquad\text{for all }\nu \in X.$$ The second equation works in the same way by looking at the derivative with respect to $\lambda$ in all directions $\mu$.