When going from the strong form of a PDE to the FEM form it seems one should always do this by first stating the variational form. To do this you multiply the strong form by an element in some (Sobolev) space and integrate over your region. This I can accept. What I don't understand is why one also has to use Green's formula (one or several times).
I've mostly been working with Poisson's equation, so if we take that (with homogenous Dirichlet boundary conditions) as an example, i e
$$ \begin{align} -\nabla^2u &= f,\quad u\in\Omega \\ u &= 0, \quad u\in\partial\Omega \end{align} $$
then it is claimed that the correct way to form the variational form is
$$ \begin{align} \int_\Omega fv\,\mathrm{d}\vec{x} &= -\int_\Omega\nabla^2 uv\,\mathrm{d}\vec{x} \\ &=\int_\Omega\nabla u\cdot\nabla v\,\mathrm{d}\vec{x} - \int_{\partial\Omega}\vec{n}\cdot\nabla u v\,\mathrm{d}\vec{s} \\ &=\int_\Omega\nabla u\cdot\nabla v\,\mathrm{d}\vec{x}. \end{align} $$
But what stops me from using the expression on the first line, isn't that also a variational form that can be used to get a FEM form? Isn't it corresponding to the bilinear and linear forms $b(u,v)=(\nabla^2 u, v)$ and $l(v)=(f, v)$? Is the problem here that if I use linear basis functions (shape functions) then I'll be in trouble because my stiffness matrix will be the null matrix (not invertible)? But what if I use non-linear shape functions? Do I still have to use Green's formula? If I don't have to: is it advisable? If I don't, do I then have a variational-but-not-weak formulation?
Now, let's say that I have a PDE with higher order derivatives, does that mean that there are many possible variational forms, depending on how I use Green's formula? And they all lead to (different) FEM approximations?