Consider this problem:
\begin{equation} \begin{array}{ll} \text{minimize } & f(x) \\ \text{subject to } & a \leq x \leq b \end{array} \end{equation}
where $a,b,x \in \mathbb{R}^n$ and $f: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is continuously differentiable. By this definition, the feasible set is closed and convex.
I would greatly appreciate it if you could give me a big picture of the classes of methods that solve this problem. I know there are some methods such as projected gradient/Newton methods or primal-dual active set/semismooth Newton methods and some variations of these methods. However, these are specific. The book by Nocedal and Wright seems to discuss the following types of methods:
- active set
- interior-point
- gradient projection
- penalty and augmented Lagrangian
- sequential quadratic programming
The book "Optimization with PDE Constraints" by Hinze, Pinnau, and Ulbrich discusses:
- Newton's methods
- Semismooth Newton's methods (which Google Scholar says is equivalent to the primal-dual active set methods)
- Sequential quadratic programming methods
I am doing a literature review of the methods for this problem and hope that through my literature review I can discover something that needs improving. I have only 2 months (January and February) to do this. I am very new to this field and do not consider myself mathematically mature. I have very little experiences. Hence, it would be great if some of you could kindly share your experiences/visions. It would help me a lot if I could have a big picture of the methods. I would forever be grateful.
