Generally speaking, I've heard numerical analysts utter the opinion that

"Of course, mathematically speaking, time is just another dimension, but still, time is special"

How to justify this? In what sense is time special for computational science?

Moreover, why do we so often prefer to use finite differences, (leading to "time-stepping"), for the time dimension, while we apply finite differences, finite elements, spectral methods, ..., for the spatial dimensions? One possible reason is that we tend to have an IVP in the time dimension, and a BVP in the spatial dimensions. But I don't think this fully justifies it.

up vote 23 down vote accepted

Causality indicates that information only flows forward in time, and algorithms should be designed to exploit this fact. Time stepping schemes do this, whereas global-in-time spectral methods or other ideas do not. The question is of course why everyone insists on exploiting this fact -- but that's easy to understand: if your spatial problem already has a million unknowns and you need to do 1000 time steps, then on a typical machine today you have enough resources to solve the spatial problem by itself one timestep after the other, but you don't have enough resources to deal with a coupled problem of $10^9$ unknowns.

The situation is really not very different from what you have with spatial discretizations of transport phenomena either. Sure, you can discretize a pure 1d advection equation using a globally coupled approach. But if you care about efficiency, then the by far best approach is to use a downstream sweep that carries information from the inflow to the outflow part of the domain. That's exactly what time stepping schemes do in time.

  • That's a good point... memory is definitely a major constraint! :) – Paul Jul 9 '12 at 17:05
  • I definitely see the point that causality comes naturally with finite differences, but not with "global coupling". Conversely, "shooting methods" for solving BVPs sort of do the opposite. It introduces unwanted causality. Analytically speaking, for certain equations (eg. 2nd order hyperbolic PDEs) causality is needed for uniqueness. However, in some cases, it is not, and I guess then one may very well do spectral methods in time as well. As you say, I think reducing the size of the system is also a big one. And it makes more sense to do FD in time than in some arbitrary spatial dimension. – Patrick Jul 11 '12 at 10:22

Similar to the causality Wolfgang mentioned in his post, we could see the reason why time dimension is special from Minkowski spacetime point of view:$ \newcommand{\rd}{\mathrm{d}} $

The $(3+1)$-dimensional spacetime has an inner product defined as $$ (A,B) = A_x B_x + A_y B_y + A_z B_z - \dfrac{1}{c^2}A_t B_t $$ if $A$ and $B$ are two 1-form in Minkowski spacetime: $A = A_x \rd x + A_y \rd y + A_z \rd z + A_t \rd t$, $B$ is defined in a similar fashion, the intuition behind defining an inner product(or rather to say, metric) is to imposing the idea of absolute light speed, such that two different points(events) in the spacetime have zero distance(happens at the "same time", like we are observing the motion of galaxies billions of lightyears away as if they are moving right now) if they are on the same light cone.

As you can see, this inner product is not positive definite due to the presence of the time dimensional scaled by the light speed $c$, therefore intuitively speaking, when treating a problem concerning a quantity propagating in the spacetime, we cannot simply apply theorems in 3-dimensional Euclidean metric to a $(3+1)$-dimensional spacetime, just think of 3-dimensional elliptic PDE theories and their corresponding numerical methods differ drastically from the hyperbolic PDE theories.

Maybe off-topic, but another major difference of space vs spacetime(elliptic vs hyperbolic) is that most elliptic equations model the equilibrium and ellipticity gives us "nice" regularity, while there are all kinds of discontinuities in hyperbolic problems(shock, rarefaction, etc).

EDIT: I don't know there is a dedicated article about the difference other than giving you the definition, based what I learned before, typical elliptic equation like Poisson equation or elasticity, models a static phenomenon, has "smooth" solution if data and boundary of domain of interest are "smooth", this is due the ellipticity(or rather to say positive definite property) of the governing differential operator, this type of equations leads us to a very intuitive Galerkin type approach(multiply a test function and integration by parts), typical continuous finite element works well. Similar things apply to parabolic equation like heat equation, which is essentially an elliptic equation marching in time, has a similar "smoothing" property, an initial sharp corner will be smoothened out over time, we call this "diffusive" or "dissapative".

For a hyperbolic problem, normally derived from a conservation law, is "conservative" or "dispersive". For example, linear advection equation, describing the certain quantity flows with a vector field, conserves how this specific quantity is like initially, just it moves spacially along this vector field, the discontinuities will propagate. Schrodinger equation, another hyperbolic equation, however, is dispersive, it is the propagation of a complex quantity, a non-oscillatory initial state will become different oscillatory wave packets over time.

As you mentioned "time-stepping", you could think the quantity "flows" in the time "fields" with a certain velocity as the causality, very similar to the linear advection equation BVP, we only have to impose the inflow boundary condition, ie, what the quantity is like when flowing into the domain of interest, and the solution would tell us what the quantity is like when flowing out, an idea very similar to every method that uses time-stepping. Solving a 2D advection equation in space is like solving a 1D one-sided propagation problem in spacetime. For numerical schemes, you could google about spacetime FEM.

  • I must say that most of what you say is above my head. But the last paragraph was very interesting, and definitely lends insight. Do you have a link to (space and spacetime) vs (elliptic and hyperbolic)? – Patrick Jul 11 '12 at 10:08
  • @Patrick Thanks for the interest, I have edited more into my answer. – Shuhao Cao Jul 11 '12 at 17:57

While there are some exceptions (e.g. fully discrete finite element methods), temporal discretization generally implies an inherently sequential dependence in flow of information. This dependency restricts semi-discrete algorithms (BVP in space, IVP in time) to compute solutions to subproblems in sequential manner. This discretization is usually preferred for its simplicity and because it offers the analyst many well-developed algorithms for higher accuracy both in space and time.

It is possible (and simpler) to use finite differences in spatial dimensions as well, but finite element methods offer easier flexibility in the type of domain on interest (e.g. non-regular shapes) than finite difference methods. A "good" choice of spatial discretization is often very problem dependent.

Your Answer


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.