# Numerical representation of linear spaces

A linear space in mathematics is a set whose elements (vectors) have operations of addition and multiplication by a scalar defined in such a way that certain properties are satisfied (commutativity, associativity etc.), and the result of those operations belongs to the same linear space, i.e., the linear space is closed. There is lots of theory for linear spaces: linear equations, eigenvalue problems etc., and all that is completely abstract and general, independent of the nature of the linear space. In mathematics linear spaces can be represented by geometric vectors, pairs of real numbers, complex numbers, quaternions, functions, and in many other ways. In numerical computations, one would normally think of a linear space in terms of a set of 1D arrays (of finite size), and linear operators on those vectors are represented by matrices. So the question is: Is there any other way to represent a linear space numerically, something that would be substantially distinct from a space of 1D finite-size arrays?

• Unless your linear space is built over a finite field it would have infinite elements. So, I guess that there will always be elements of a linear space (over the reals) that can't be represented numerically. Jun 9 at 19:10

This sounds like an exercise in semiotics, but I'll try all the same. One alternative representation scheme for the particular case of the space of continuous functions on some n-dimensional domain $$\Omega$$ would be to use a context-free language. While $$C^0(\Omega)$$ is a linear space, you could instead imagine (1) starting with a set of terminal symbols consisting of real constants and the values of the coordinates $$x_1, \ldots, x_n$$ of points in the domain, all of which are continuous functions on $$\Omega$$ and (2) picking a set of operations that preserve continuity, enable you to do interesting things, and which enables you to approximate any continuous function. A vague impersonation of a BNF grammar for the language is:

R := <any real number>
C := x_1, ..., x_n
E := R | C | exp(E) | sin(E) | cos(E) | abs(E) | E + E | E - E | E * E


You can verify that the non-terminal symbols all give continuous expressions if their arguments do. The Stone-Weierstrass theorem tells us that elements of this "language" get arbitrarily close to any continuous function. I included the transcendental and absolute value functions because they make it so that the representation doesn't trivially map onto polynomials and thus arrays. This isn't completely in the spirit of your question because I'm taking advantage of special properties of $$C^0$$ like the fact that it's an algebra and not just a vector space.

Nonetheless, by far the most common representation for elements of a vector space will, under the hood, be an array of numbers. It is a good question to ask what that array of numbers means and two arrays of the same shape don't necessarily mean the same thing. For example, consider the array that represent the expansion coefficients with respect to some Galerkin basis for the Sobolev space $$H^1(\Omega)$$. This array has the same shape as the array of expansion coefficients of a linear functional that lives in the dual space $$H^{-1}(\Omega)$$. We won't get a type or shape error if we add those arrays, but that doesn't mean that the sum is a sane element of $$H^1(\Omega)$$. You run into this subtlety when using the adjoint method for PDE-constrained optimization.

• That's a good thought, thanks! Along the same lines, can we say that functions in a symbolic system like Mathematica (represented by human-readable strings) is a linear space? Say, we have objects there represented by strings like "Sin[x]", "Exp[x]+2Tanh[x]", but those objects there have the meaning of functions in the domain (−∞,∞), and we have there the operations of addition and multiplication by a scalar - so that is a linear space? I guess under the hood ultimately it will be arrays of numbers in a digital computer, not sure if for an analog computer we could make the same statement. Jun 10 at 0:00
• Yea this would be how expressions are represented in a computer algebra system. But in a real CAS things are more complicated because you have division, which might break continuity, and maybe conditional expressions too. Under the hood it probably isn't an array of numbers but rather a proper tree or S-expression or some such. Jun 10 at 0:14