Similar to the causality Wolfgang mentioned in his post, we could see the reason why time dimension is special from Minkowski spacetime point of view:$
\newcommand{\rd}{\mathrm{d}}
$
The $(3+1)$-dimensional spacetime has an inner product defined as
$$
(A,B) = A_x B_x + A_y B_y + A_z B_z - \dfrac{1}{c^2}A_t B_t
$$
if $A$ and $B$ are two 1-form in Minkowski spacetime:
$A = A_x \rd x + A_y \rd y + A_z \rd z + A_t \rd t$, $B$ is defined in a similar fashion, the intuition behind defining an inner product(or rather to say, metric) is to imposing the idea of absolute light speed, such that two different points(events) in the spacetime have zero distance(happens at the "same time", like we are observing the motion of galaxies billions of lightyears away as if they are moving right now) if they are on the same light cone.
As you can see, this inner product is not positive definite due to the presence of the time dimensional scaled by the light speed $c$, therefore intuitively speaking, when treating a problem concerning a quantity propagating in the spacetime, we cannot simply apply theorems in 3-dimensional Euclidean metric to a $(3+1)$-dimensional spacetime, just think of 3-dimensional elliptic PDE theories and their corresponding numerical methods differ drastically from the hyperbolic PDE theories.
Maybe off-topic, but another major difference of space vs spacetime(elliptic vs hyperbolic) is that most elliptic equations model the equilibrium and ellipticity gives us "nice" regularity, while there are all kinds of discontinuities in hyperbolic problems(shock, rarefaction, etc).
EDIT: I don't know there is a dedicated article about the difference other than giving you the definition, based what I learned before, typical elliptic equation like Poisson equation or elasticity, models a static phenomenon, has "smooth" solution if data and boundary of domain of interest are "smooth", this is due the ellipticity(or rather to say positive definite property) of the governing differential operator, this type of equations leads us to a very intuitive Galerkin type approach(multiply a test function and integration by parts), typical continuous finite element works well. Similar things apply to parabolic equation like heat equation, which is essentially an elliptic equation marching in time, has a similar "smoothing" property, an initial sharp corner will be smoothened out over time, we call this "diffusive" or "dissapative".
For a hyperbolic problem, normally derived from a conservation law, is "conservative" or "dispersive". For example, linear advection equation, describing the certain quantity flows with a vector field, conserves how this specific quantity is like initially, just it moves spacially along this vector field, the discontinuities will propagate. Schrodinger equation, another hyperbolic equation, however, is dispersive, it is the propagation of a complex quantity, a non-oscillatory initial state will become different oscillatory wave packets over time.
As you mentioned "time-stepping", you could think the quantity "flows" in the time "fields" with a certain velocity as the causality, very similar to the linear advection equation BVP, we only have to impose the inflow boundary condition, ie, what the quantity is like when flowing into the domain of interest, and the solution would tell us what the quantity is like when flowing out, an idea very similar to every method that uses time-stepping. Solving a 2D advection equation in space is like solving a 1D one-sided propagation problem in spacetime. For numerical schemes, you could google about spacetime FEM.
