Recently, I saw this question: how to measure the error of a finite difference method

I am student of simulation sciences and unfortunately, for me, it's totally unclear, what norm to use in what context.

Quite often, we use the Euclidian norm or the L2 norm, but why does one choose different norms, what's their meaning besides the numerical / mathematical definition? Or more precise: What is the reason to use a specific norm in a specific context?

  • 2
    $\begingroup$ This is a huge question. Are you interested in norms for measuring errors in numerical solutions of differential equations? If so, you should narrow the scope of the question. $\endgroup$ Jul 15 '12 at 12:51
  • $\begingroup$ At the moment, we are calculating the solutions of simple PDEs like the Poisson equation. But my question is not focused on that. I want to learn, how to use norms in general. $\endgroup$
    – vanCompute
    Jul 15 '12 at 13:04
  • 2
    $\begingroup$ From the FAQ: Your questions should be reasonably scoped. If you can imagine an entire book that answers your question, you’re asking too much. $\endgroup$ Jul 15 '12 at 14:58
  • $\begingroup$ My problem is, that I cannot even image an antire book with the answer to that question. It's somehow a missing piece/gap in my knowledge. The answer to that problem can't be that big, as everybody uses norms. If you see the chance of narrowing the question to make it more precise, than I am open to that. I will reformulate the question a bit, I hope it is better then. $\endgroup$
    – vanCompute
    Jul 16 '12 at 8:02
  • $\begingroup$ There are many. Here's one: books.google.co.uk/books/about/… $\endgroup$ Jul 16 '12 at 9:00

For measuring the error in the solution of PDE, it is quite natural to choose the norm of the space in which the solution lies. For example, for elliptic PDEs, the solution lies in $H^1$ and so it is natural to choose the $H^1$ norm to measure the error. This makes sense because, for example, the solution does not lie in the space $W^{1,\infty}$ and so it does not make sense to compute the maximum error in the gradient simply because you can't measure this error if even the exact solution has points where the gradient is not finite. In other words, it doesn't make sense to measure the error in the norm of a space $X$ (e.g., $X=W^{1,\infty}$) if the exact solution lies in $Y$ (e.g., $Y=H^1$) and $X\subset Y$.

On the other hand, we frequently measure the error in the norm of a space $Z$ if $Z\supset Y$, e.g., when we measure the error in $L_2$. For these other norms, it is sometimes because of physical importance but equally often simply a matter of convenience. The $L_2$ norm sometimes has some physical meaning: for example, the integral of the square of the electric field $\int E(x)^2 \; dx$, i.e., the square of the $L_2$ norm, is the energy stored in the electric field; likewise, the square of the norm of the solution of the wave equation is the potential energy stored in the solution. At other times, it is just a conveniently chosen norm. For example measuring the $L_2$ norm of the error in the time-dependent heat equation is almost always the wrong choice since the physically relevant quantities (the total thermal energy, the amount of material) is in reality the $L_1$ norm of the solution; in this case, measuring the error in the $L_2$ norm has no meaning other than being convenient.

  • $\begingroup$ This is an important piece in the puzzle, thank you! What do you mean exactly with the argument the $L2$ norm is more convenient than the $L1$ norm? $\endgroup$
    – vanCompute
    Jul 16 '12 at 7:55
  • 1
    $\begingroup$ For example, the $L_2$ norm of a finite element field can be computed as $U^T M U$ where $M$ is the mass matrix and $U$ is the vector of nodal values. This is just a convenient way of computing things if you happen to have the mass matrix lying around already (e.g., in a time dependent problem), whereas for the $L_1$ norm you have to do an extra assembly loop. $\endgroup$ Jul 16 '12 at 17:02

The Euclidean norm is often used based on the assumption that the Euclidean distance of two points is a reasonable measure of distance. But unless this is the case, this choice is not preferable to a problem-adapted choice. For example if the typical size of the components of a vector are very different (since they mean very different things), the Euclidean norm is very poor as it hardly takes into account the effects of changes in the small-size components. In such a case, one either needs to first scale the vectors to have similar sized components before applying norms, or one must use a norm that scales different components differently.

The norm $\|x\|$ of a vector $x$ (and similarly for matrices and functions) is a measure of its size; this measure must be adapted to the meaning of the problem you are solving. In finite dimensions, all norms are equivalent, in the sense that they describe the same topology; but the numerical values may depend quite a lot on the particular norm. (For the topology, the only thing of interest is the limit. In finite D this is independentent of the norm, i.e., $\|x_k−x\|\to 0$ in any norm if $\lim x_k=x$. But how close one is to the limit depends a lot on in which norm you measure it.)

Therefore one must choose a meaningful norm to get meaningful results.

In infinite-dimensional spaces (which in particular includes the common function spaces), norms are no longer equivalent, and different norms may lead to different topologies. Now one must choose a suitable norm even to get finite results, and bounding terms may be impossible wwithout a good choice of the norm.

As an exercise, I'd like to suggest that you compare the values of the $p$-norm for $p=1,2,\infty$ for a variety of vectors in $R^n$ parameterized by $n$, and do the same in various spaces of sequences $x=(x_1,x_2,\dots)$. You'll then appreciate the differences. A good example is the vector with $i$the entry $x_i=\epsilon/i^s$, where $s>0$. Here for tiny $\epsilon$ and large $n$ (approximate the sum by an integral) $\|x\|_p\approx \epsilon\frac{1-1/n^{ps-1}}{ps-1}$, which becomes infinitely large as $n\to\infty$ when $p\le 1/s$ but remains tiny when $p>1/s$.

  • $\begingroup$ What do you mean by topology? I only think of network topologies, but this is not intended, I think. $\endgroup$
    – vanCompute
    Jul 15 '12 at 13:09
  • $\begingroup$ @vanCompute en.wikipedia.org/wiki/Normed_vector_space#Topological_structure $\endgroup$
    – Jed Brown
    Jul 15 '12 at 13:30
  • 1
    $\begingroup$ I think the equivalence of norms in finite-dimensions is very misleading in the numerical context, since the comparision constants of the prominent norms blow as the dimension increases. $\endgroup$
    – shuhalo
    Jul 15 '12 at 13:53
  • $\begingroup$ I don't grasp the idea of topology yet: All norms on a finite-dimensional vector space are equivalent from a topological viewpoint as they induce the same topology. But what to understand under topology: en.wikipedia.org/wiki/Topology $\endgroup$
    – vanCompute
    Jul 15 '12 at 14:01
  • $\begingroup$ @Martin: This is just my point, and the exercise at the end demonstates it. $\endgroup$ Jul 15 '12 at 14:01

A few remarks:

In general, which norm you choose depends on what you want to measure. It is that simple.

For numerical pde, the $L^2$-norm has the convenient property of providing a Hilbert space structure. A natural reason to use this norm comes from the treatment of measurement errors, as described in https://scicomp.stackexchange.com/a/2763/238. I do not know whether there are other reasons that lie beyond mathematical feasibility.

The $L^\infty$-norm is used when you want to assert a maximum bound on the error "pointwise". It is natural to represent the dual space of $L^\infty$ by functions with finite $L^1$-norm.

Other $L^p$-norms are used in nonlinear PDE, and Sobolev norms are the simple generalization of $L^p$-spaces if you want to control a function and its generalized derivatives.

  • $\begingroup$ Ok, sounds a little bit tricky. It seems to be only applicable if the "measurement satisfies intuitive probability laws" link. $\endgroup$
    – vanCompute
    Jul 16 '12 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.