Recently, I saw this question: how to measure the error of a finite difference method
I am student of simulation sciences and unfortunately, for me, it's totally unclear, what norm to use in what context.
Quite often, we use the Euclidian norm or the L2 norm, but why does one choose different norms, what's their meaning besides the numerical / mathematical definition? Or more precise: What is the reason to use a specific norm in a specific context?
For measuring the error in the solution of PDE, it is quite natural to choose the norm of the space in which the solution lies. For example, for elliptic PDEs, the solution lies in $H^1$ and so it is natural to choose the $H^1$ norm to measure the error. This makes sense because, for example, the solution does not lie in the space $W^{1,\infty}$ and so it does not make sense to compute the maximum error in the gradient simply because you can't measure this error if even the exact solution has points where the gradient is not finite. In other words, it doesn't make sense to measure the error in the norm of a space $X$ (e.g., $X=W^{1,\infty}$) if the exact solution lies in $Y$ (e.g., $Y=H^1$) and $X\subset Y$.
On the other hand, we frequently measure the error in the norm of a space $Z$ if $Z\supset Y$, e.g., when we measure the error in $L_2$. For these other norms, it is sometimes because of physical importance but equally often simply a matter of convenience. The $L_2$ norm sometimes has some physical meaning: for example, the integral of the square of the electric field $\int E(x)^2 \; dx$, i.e., the square of the $L_2$ norm, is the energy stored in the electric field; likewise, the square of the norm of the solution of the wave equation is the potential energy stored in the solution. At other times, it is just a conveniently chosen norm. For example measuring the $L_2$ norm of the error in the time-dependent heat equation is almost always the wrong choice since the physically relevant quantities (the total thermal energy, the amount of material) is in reality the $L_1$ norm of the solution; in this case, measuring the error in the $L_2$ norm has no meaning other than being convenient.
The Euclidean norm is often used based on the assumption that the Euclidean distance of two points is a reasonable measure of distance. But unless this is the case, this choice is not preferable to a problem-adapted choice. For example if the typical size of the components of a vector are very different (since they mean very different things), the Euclidean norm is very poor as it hardly takes into account the effects of changes in the small-size components. In such a case, one either needs to first scale the vectors to have similar sized components before applying norms, or one must use a norm that scales different components differently.
The norm $\|x\|$ of a vector $x$ (and similarly for matrices and functions) is a measure of its size; this measure must be adapted to the meaning of the problem you are solving. In finite dimensions, all norms are equivalent, in the sense that they describe the same topology; but the numerical values may depend quite a lot on the particular norm. (For the topology, the only thing of interest is the limit. In finite D this is independentent of the norm, i.e., $\|x_k−x\|\to 0$ in any norm if $\lim x_k=x$. But how close one is to the limit depends a lot on in which norm you measure it.)
Therefore one must choose a meaningful norm to get meaningful results.
In infinite-dimensional spaces (which in particular includes the common function spaces), norms are no longer equivalent, and different norms may lead to different topologies. Now one must choose a suitable norm even to get finite results, and bounding terms may be impossible wwithout a good choice of the norm.
As an exercise, I'd like to suggest that you compare the values of the $p$-norm for $p=1,2,\infty$ for a variety of vectors in $R^n$ parameterized by $n$, and do the same in various spaces of sequences $x=(x_1,x_2,\dots)$. You'll then appreciate the differences. A good example is the vector with $i$the entry $x_i=\epsilon/i^s$, where $s>0$. Here for tiny $\epsilon$ and large $n$ (approximate the sum by an integral) $\|x\|_p\approx \epsilon\frac{1-1/n^{ps-1}}{ps-1}$, which becomes infinitely large as $n\to\infty$ when $p\le 1/s$ but remains tiny when $p>1/s$.
A few remarks:
In general, which norm you choose depends on what you want to measure. It is that simple.
For numerical pde, the $L^2$-norm has the convenient property of providing a Hilbert space structure. A natural reason to use this norm comes from the treatment of measurement errors, as described in https://scicomp.stackexchange.com/a/2763/238. I do not know whether there are other reasons that lie beyond mathematical feasibility.
The $L^\infty$-norm is used when you want to assert a maximum bound on the error "pointwise". It is natural to represent the dual space of $L^\infty$ by functions with finite $L^1$-norm.
Other $L^p$-norms are used in nonlinear PDE, and Sobolev norms are the simple generalization of $L^p$-spaces if you want to control a function and its generalized derivatives.
