I have been pondering about this issue for some time...

Say, I want to minimize a costfunctional $$ \tilde J(u) = J(v(u),u) = \frac 12 \int_0^T (v-v_0)^2 + \alpha u^2 dt $$ subject to $$ \dot v = v^2 + u, \quad v(0)=0. $$ Then, from the first order necessary optimality conditions it follows that at an optimal point $(v^*,u^*)$, there is a $\lambda$ that solves $$ -\dot \lambda = 2v^*\lambda - (v^*-v_0), \quad \lambda(T)=0, \tag{1} $$ such, that that the gradient of $\tilde J$ is given as $$ D_u\tilde J(u^*) = \lambda + \alpha u^*=0. \tag{2} $$

From $(1)$ and $(2)$, I infer that $$u^*(T) = 0.$$ My Question: Is this right? Or is there something wrong in the argumentation.

Two more remarks:

  1. I don't think that the answer lies in the right functional analytic formulation. (I have had a very close look at this).
  2. If one uses gradient based methods, then $(2)$ implies that the update is zero at $t=T$. Which means that the terminal value of the converged control will equal the terminal value of the initial guess. (I have seen this in a master thesis, where a fluid/structure interaction was successfully controlled by the adjoint based approach; see the screenshot) enter image description here

EDIT: The conclusion that $u^*(T) = 0$ (as well as the update) is not true. (Thanks to L.P. for pointing this out) However, in practice, in a gradient descent method, one updates an initial guess $u_0$ via $u_1 = u_0 - s D_u\tilde J(u_0)$ so that $\lambda(T) = 0$ implies that

$$ u_1(T) = (1-s\alpha)u_0(T) $$

Note, that the step size $s=\mathcal O(1)$ whereas $\alpha$ can be like $10^{-5}$ so that a gradient iteration, in fact, hardly affects the endpoint of the control.

  • 2
    $\begingroup$ You are missing a factor of 2 in eq. (1) in front of the $v^\ast\lambda$ term as well as the last term. I think you also want to fix that the equation you have is not actually a PDE but an ODE. $\endgroup$ – Wolfgang Bangerth Apr 5 '17 at 13:38
  • $\begingroup$ @Jan it looks to me like the gradient in eq (2) should read $D_u \tilde{J} = \int_0^T \left( \lambda + \alpha u \right) dt$ $\endgroup$ – GoHokies Apr 5 '17 at 19:38
  • $\begingroup$ Thanks @WolfgangBangerth for pointing this out. (The 2 before the last term cancels with 1/2 in front of the integral in $\tilde J$). I have not specified any spaces, but I removed the reference to the PDE. $\endgroup$ – Jan Apr 5 '17 at 19:39
  • $\begingroup$ @GoHokies , there is a lot missing of the derivation. Basically, $\tilde J$ maps $u$ into the reals. Then $D_u\tilde J(u^*)$ is a functional and can be identified with a function, say $\bar u$ (since $u$ is to be assumed in a Hilbert space). Then $D_u \tilde J(u^*)u$ -- the application of the functional on a function $u$ -- would include the integral... $\endgroup$ – Jan Apr 5 '17 at 19:51
  • $\begingroup$ @Jan you're perfectly right, I mistook your $D_u \tilde{J}$ notation to mean the directional derivative (with the direction left out). $\endgroup$ – GoHokies Apr 5 '17 at 20:14

You won't like the answer, but it is in fact "This lies in the right functional-analytic framework" — which you have not given! In particular, you did not specify in which function space you are looking for $u$ in. If $u\in L^2$ as the formulation suggests, then asking for the value of $u$ at a single point makes no sense a priori. (Even if the ODE is based on classical derivatives, the optimal control will be $\bar u = \alpha^{-1}\bar \lambda$ in the sense of $L^2$. Hence even though the latter is indeed continuous and will have zero value at $t=T$, if you change $\bar u$ at $t=T$ arbitrarily, you will still get the same $L^2$ function (or rather, equivalence class of functions), so asking for the value of $\bar u(T)$ is not meaningful.)

Now if you do a naive discretization of $u,v,\lambda$ by finite differences, identifying their function with their values at the nodes, you get exactly the behavior that you describe. If you do not want that, the usual procedure is to discretize a control in $L^2$ as piecewise constant on each control interval, in which case the optimal control $\bar u$ will not be $\alpha^{-1}\bar \lambda$ but its $L^2$ projection — i.e., the average over each interval, which can be nonzero on the last interval even for $\bar \lambda(T)=0$.

You could also look for $u\in H^1$ (not $H^1_0$!) — adding an appropriate penalty to make the problem well-posed. In this case, $\bar u = (\alpha I+\beta \Delta)^{-1} \bar \lambda$ (i.e., the solution to $\beta\Delta u + \alpha u =\bar \lambda$ subject to homogeneous Neumann boundary conditions) will be continuous due to the Sobolev embedding and have a well-defined — not necessarily zero — value at $t=T$ even if $\bar \lambda(T)=0$.

  • $\begingroup$ Thanks, I do like this answer. Which I understand as "The derivation is OK, but the zero terminal value is not significant for the control". $\endgroup$ – Jan Apr 10 '17 at 7:53
  • $\begingroup$ One more thing. If $u=\alpha^{-1}\lambda$ and $\lambda$ is continuous (as the solution of an ODE), than a different terminal value $\lambda(T)$ for the ODE will give a different $u$ -- also in the $L^2$ norm. Thus, I would argue that $\lambda(T)=0$ has an effect on $u$ also in the $L^2$ sense but (probably) no effect on the costfunction. $\endgroup$ – Jan Apr 10 '17 at 8:08
  • 2
    $\begingroup$ @Jan I'd rather formulate it as "The derivation is formally correct, but you need to be careful in interpreting it, especially going from the continuous to the discrete level." And you are of course correct that a different $\lambda$ would give a different $u$, and that the zero terminal value has an effect (and makes sense intuitively: it can't be optimal to put in work at the end of the time interval when there's no time left to influence the solution); the point is just that the equation $\bar u = \alpha^{-1}\bar \lambda$ does not mean that $\bar u$ and $\bar \lambda$ are the same thing. $\endgroup$ – Christian Clason Apr 10 '17 at 8:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.