Given data $d$ on boundary, the material parameter $\beta$ you would like to estimate, the forward problem is: $$\mathcal{A}(\beta) u = f $$ where the differential operator $\mathcal{A}$ depends on the parameter, andalso accompanied with some boundary condition(see Remark 1). Forward problem actually can be ill-posed.
Now your inverse problem would be using this PDE as a constraint, to solve a functional minimization problem like the following: $$ \min_{u,\beta} \mathfrak{F}(u,\beta,d,u_h,\beta_h) $$ for example, $\mathfrak{F}$ canis your objective functional, could be the Tikhonov functional, and others.(see Remark 2)
If you would like to do some theoretical work on inverse problems, Functional Analysis is a must, because you will use Fréchet derivatives as your daily snacks. Also some PDE theory is needed. For this purpose, reading Lars Hörmander's four-volume series The Analysis of Linear Partial Differential Operators would be more than nice, but it costs too much time. A rather concise but difficult to read book is Victor Isakov's Inverse problems for partial differential equations, btw despite the fact that Isakov works in a not-so-famous university, he is always considered to be one of the great mathematician working on inverse problems by Gunther Uhlmann, Stanley Osher, etc.(more book recommendation see Remark 3)
Since you mentioned finite element, I am guessing you would like to learn this topic from a more numerical point of view, then Stig Larsson's book Partial Differential Equations with Numerical Methods is not hard to read even you are not a numerical analyst, it would get you started in finite element methods. For the connection between the finite element methods and inverse problems, there isn't a dedicated book about this that I know of, however there are many research articles on this. For example the current development in inverse problems Gunther Uhlmann edited: Inside Out: Invserse Problems and Applications shall be a neat introduction to some advanced research area.
To sum up, my suggestion would be: Know what exactly the PDE constraint optimization you are gonna do first, do not bother with the lengthy mathematical theory behind it, then go learning the corresponding finite element methods for the forward problem, lastly go back to the inverse problem to see what you need there.
SOME UPDATES
Remark 1: If we just have a single boundary measurement, we could treat the data $d$ as the Dirichlet boundary condition for the forward problem; If what we know is a mapping(!), aka Dirichlet-Neumann map, ie given input electromagnetic wave of certain frequency, we could get certain feedback on the boundary, then the boundary data $d$ is not the boundary condition of the forward problem, but rather that Dirichlet-Neumann map, working like a black-box.
Remark 2: A potential difficulty here, due to the property of high frequency waves, the objective functional may have lots and lots of local minima, and this problem is still an open research area.
Remark 3: I consulted my officemate today who is working on Dirichlet-Neumann mapping, and he recommended two more books, the first one is the yet to be published lecture notes by Gunter Uhlmann and Joel Feldman, this is a more user-friendly read than Isakov's. Another book is by Colton and Kress, Inverse Acoustic and Electromagnetic Scattering Theory, which is even more user-friendly assuming you know some PDE and/or functional analysis and/or finite element.