# Motivation behind Collocation Method

In the previous question "Motivation behind Galerkin method", Paul gives a good and easy-to-understand explanation indicating that the Galerkin method is a kind of projection method. Can anyone explain why the Collocation method also belongs to the projection methods?

I find a reference "Projection methods and the curse of dimensionality" and know the collocation method employs the Dirac delta function as the weight function.

• You basically answer your own question in the last sentence. Can you go into a bit more detail why this isn't enough for you? Jun 23, 2016 at 11:00
• Well, Galerkin method is a kind of projection method because it makes the residual value of the equation orthogonal to the space where we want to find the solution. But collocation method sets the residual value vanished on several points. Then what's the geometrical meaning behind this method?
– Jay
Jun 23, 2016 at 11:10

The idea behind Galerkin methods is the following: Assume you have an equation $F(x) = 0$ for some (possibly nonlinear) operator $F:X\to Y$ between (infinite-dimensional) spaces $X$ and $Y$, i.e., you are trying to find $\bar x\in X$ with $F(\bar x)=0$. It is a theorem in functional analysis that $F(x) = 0$ if and only if $$\langle F(x),y^*\rangle_Y = 0\qquad\text{for all } y^*\in Y^*,$$ which is the dual space of $Y$. (Bear with me, there will be examples later.) This is still something you can't solve, but if you replace $X$ by a finite-dimensional subspace $X_h\subset X$ and $Y^*$ by a finite-dimensional subspace $Y^*_h\subset Y^*$, you can try to find $\bar x_h \in X_h$ such that $$\label{1} \langle F(\bar x_h),y^*_h\rangle_Y = 0\qquad\text{for all } y^*_h \in Y_h^*, \tag{1}$$ or equivalently, since $Y^*_h$ is finite-dimensional, for all basis functions $y^*_1,\dots,y^*_n$ of $Y^*_h$. If $X_h$ and $Y_h$ are chosen correctly, this gives you $n$ independent conditions for $n$ unknown coefficients.
Let's specialize a bit: If $Y=X$ is a Hilbert space, you can take $Y^*_h = X_h$; this is called a Ritz-Galerkin method. Furthermore, if $F(x)=x-z$ for some $z\in X$, what we're actually doing is looking for $x_h\in X_h$ such that $x_h-z$ is orthogonal to $X_h$ -- these are exactly the conditions that characterize the orthogonal projection of $z$ onto the subspace $X_h$. Hence the name (generalized) projection method.
Now for collocation (and the promised example): If $Y\neq X$ and thus $Y_h^* \neq X_h$, one speaks of Petrov-Galerkin methods. In the specific case that $Y=C(E)$, the space of continuous functions on some closed set $E$ (e.g., the real interval $[0,1]$), the dual space is given by the space $\mathcal{M}(E)$ of Borel measures on $E$, and the duality pairing between a continuous function $u$ and a measure $\mu$ is given by $$\langle u,\mu\rangle_C = \int_E u(x) \,d\mu(x).$$ An important example of Borel measures are the Dirac measures $\delta_{x_0}$ for $x_0\in E$, which are defined by their action as $$\langle u,\delta_{x_0} \rangle_C := u(x_0).$$ Now we're almost there: If we take $Y^*_h$ as the subspace spanned by Dirac measures for $n$ distinct points $x_1,\dots,x_n\in E$, the Galerkin conditions \eqref{1} become $$F(\bar u_h)(x_i) = \langle F(\bar u_h),\delta_{x_i}\rangle_Y = 0\qquad\text{for all } 1\leq i \leq n.$$ Similarly, if $X=C(D)$ for some space $D$ (e.g., $D=[0,1]$ as well), you can choose $X_h$ as the subspace of piecewise linear (on a given set of intervals $[x_1,x_2],\dots,[x_{n-1},x_n]\subset D$ and continuous functions on $D$ and uniquely represent $\bar u_h\in X_h$ by its values $\bar u_h(x_i)$ for $1\leq i\leq n$. Then what you actually have to find is the values $\bar u_h(x_i)$ for $1\leq i\leq n$ such that $F(\bar u_h)(x_i) = 0$ for all $1\leq i\leq n$ -- which is precisely collocation.
The relationship can be seen without getting into functional analysis. When well formulated, collocation methods approximate the integrals in other projection methods with numerical quadrature. This leads to a method afflicted by name proliferation - orthogonal collocation, pseudospectral, differential quadrature, GNI.... The Galerkin method is approximated by collocation at Lobatto points, the method of moments or Legendre-tau method by collocation at Gauss points, and a sort of Galerkin method with additional weight $$1/{\sqrt{1-x^2}}$$ by collocation at Chebyshev points. Due to this relationship, the collocation methods can also be considered projection methods or orthogonalization methods. I touched on this recently in another discussion - Role of weight function in Galerkin methods. It is discussed more fully in the monograph I'm working on at http://TildenTechnologies.com/Numerics.