13
$\begingroup$

Cross-posted on Math.SE

Are there real-world applications that call specifically for eigenvalues rather than singular values?

I often see eigendecomposition used as "poor-man's SVD" For instance it's used in Matlab's Lyapunov solver, but that could be reformulated in terms of SVD with greater cost ($22n^3$ instead of $9n^3$, Higham's big six). Similarly, PCA can be done using SVD.

The issue I see is that eigenvalues can be numerically unstable, which is a sign that you don't actually need to know their values, but instead need a derived statistic like spectral radius. Eigenvalue decomposition at $9n^3$ ops seems like an overkill to compute spectral radius.

$\endgroup$
9
  • 5
    $\begingroup$ A very similar version of this question was posted to MSE and there is some additional discussion there: math.stackexchange.com/questions/4830064/… $\endgroup$
    – whpowell96
    Dec 19, 2023 at 0:29
  • $\begingroup$ To comment on your last point: There are methods to compute (or estimate) the spectral radius that do not involve finding all the eigenvalues. For example, (the power iteration)[en.wikipedia.org/wiki/Power_iteration] and the Rayleigh quotient. Finding the spectral radius can be extremely useful sometimes: Say you are solving a linear time-dependent PDE using FEM, you end up with a large system of ODEs $\mathbf{u}'=A\mathbf{u}$. Before throwing a explicit solver, we need to estimate the spectral radius. $\endgroup$
    – Tulip
    Dec 19, 2023 at 1:30
  • $\begingroup$ Are you sure Lyapunov equations can be solved with an SVD? It doesn't seem obvious to me. The usual algorithm relies on the Schur decomposition, which has the same change of basis on the left and on the right, and that looks like it's a necessary feature to preserve. $\endgroup$ Dec 19, 2023 at 7:32
  • $\begingroup$ Also, note that eigenvalues of nonsymmetric matrix are typically computed with a Schur decomposition, not a full eigendecomposition. Are you interested in applications of eigendecompositions specifically, or are those that rely on the Schur decomposition ok? $\endgroup$ Dec 19, 2023 at 7:35
  • 1
    $\begingroup$ @YaroslavBulatov That's not an SVD, that's an eigendecomposition. An eigendecomposition is guaranteed to return $V=U$, an SVD is not. For positive definite matrices (and for that case only) an orthogonal eigendecomposition is also an SVD, but the reverse isn't true. $\endgroup$ Dec 19, 2023 at 7:49

8 Answers 8

12
$\begingroup$

You can compute analytic functions of matrices using the eigendecomposition (or more generally by using the Jordan normal form in case the matrix is defective), you cannot do so with the singular value decomposition. E.g. say you have an analytic function $f$ then you can write it as a series $f(x) = \sum_{k=0}^{\infty} c_k x^k$. Now let $A$ be a square matrix, then you can define $f(A) = \sum_{k=0}^{\infty} c_k A^k$. If you have the eigendecomposition of $A = PDP^{-1}$ then substituting in the series yields $f(A) = Pf(D)P^{-1}$, i.e. $f$ is just applied to the eigenvalues (it's a bit trickier for defective matrices, where you need to use the Jordan normal form). The same doesn't apply to the SVD unless $U=V$ as otherwise those matrices cannot cancel out like $P$ and $P^{-1}$. A common application of this is the solution of matrix ODEs with constant coefficients: $$\partial_t u(t) = Au(t) \implies u(t) = \exp(tA)u(0).$$ If you know the eigendecomposition this is easily evaluated even along positive eigenvalues where time stepping methods behave very poorly due to the exponential growth. Such matrix ODEs often arise from a spatial discretization of a time-evolution PDE. If your spatial differential operator is self-adjoint (e.g. $A\approx \Delta$) then the matrix $A$ is symmetric/Hermitian.

As far as estimation of the spectral radius goes, there are much more efficient methods than a full eigendecomposition. Yousef Saad has a whole book on sparse eigenvalue problems if you're interested. Also as a cheap upper bound you can use the Gershgorin circle theorem.

$\endgroup$
9
$\begingroup$

In Quantum theory the observables corresponding to an operator are the eigenvalues of that operator. So, as an example, should you want the energy levels available to electrons in a molecule you need to diagonalise an (approximation of the) operator corresponding to that, namely the Hamiltonian.

$\endgroup$
7
$\begingroup$

The SVD is a special case of the eigen-decomposition, or could be thought closely related to it.

For instance, the Kahan-Golub algorithm to compute the SVD is developed from the eigen-decomposition of the symmetric block-matrix $$\pmatrix{0&A\\A^T&0}.$$ The structure of this matrix is used to shrink the dimension of this square matrix from the sum of the dimensions of $A$ back to the minimum of the dimensions of $A$. This is the first reduction that in the eigen-decomposition in the generic matrix case is the transformation to Hessenberg form.

$\endgroup$
6
$\begingroup$

The eigenvalues of partial differential operators describing mechanical or electromagnetic systems are related to the resonance frequencies. For example, the frequencies at which a drum or guitar or string instrument vibrates are the square roots of the eigenvalues of the Laplace operator. The frequencies at which a building or bridge sways are the square roots of the eigenvalues of the linear elasticity operator. The frequencies at which an electromagnetic cavity (say, in your microwave oven, or in the particle accelerators used for medical cancer therapy devices) oscillates are the square roots of the eigenvalues of the Maxwell operator. There are many practical applications in which knowing these resonance frequencies is important, typically because you want that a device/instrument/building does or does not have specific resonant frequencies.

In order to compute the eigenvalues of these operators, you "discretize" them to obtain a finite-dimensional matrix, and then you compute the eigenvalues of this matrix. In many cases, these matrices have sizes ranging in the hundreds of thousands to the hundreds of millions.

$\endgroup$
5
  • $\begingroup$ Is this matrix not normal? Is eigenvalue (in)stability/non-Lipschitzness not a problem there? $\endgroup$ Dec 19, 2023 at 16:27
  • 1
    $\begingroup$ In my experience, nonnormality is rare in PDEs. The key is that eigenvalues tell you more interesting things than singular values for these problems, and the problems are so large that resources may dictate choosing only one. $\endgroup$
    – whpowell96
    Dec 19, 2023 at 17:44
  • $\begingroup$ The matrices that result from this are typically positive semidefinite (in the undamped case), or at least not far from it (with some physical damping/attenuation of oscillations). The eigenvalues are typically stable, except if you are at a point where two eigenvalues cross over as a function of external parameters (cavity size, material constants, etc.). $\endgroup$ Dec 19, 2023 at 22:42
  • $\begingroup$ @whpowell96 Off the top of my head: transport pdes and osomosis or other pdes with a drift term result in a non-symmetric system so I do not think it is that rare. $\endgroup$
    – lightxbulb
    Dec 19, 2023 at 23:16
  • $\begingroup$ Ah you're right. I meant to say non-diagonalizability. Although I don't think I've seen a case where the eigenbasis from a PDE has had probmlematically high condition number. $\endgroup$
    – whpowell96
    Dec 19, 2023 at 23:26
6
$\begingroup$

The energy levels available to a system (e.g. an atom, molecule, material, etc.) are the eigenvalues of the system's Hamiltonian matrix.

The following diagram which is presented to grade 9 (typically aged 13 to 15) students in the Ontario curriculum, shows labels four different energy levels, which correspond to the lowest four eigenvalues of the atom's Hamiltonian matrix:

~~ ~~ ~~ ~~ ~~ ~~enter image description here

Therefore, all of spectroscopy is about eigenvalues and the differences between them.

It is how we know that there's water on Mars, and CO2 on Venus and how we know the composition of stars and how we know the composition of the universe:

~~ ~~ enter image description here

We also use spectroscopy to check for pollutants in fuels, to check whether or not currency is counterfeit, and we use it in medical, geological, and atmospheric/climate applications among many, many other things.

The eigenvalues of the H atom within a non-relativistic model of the universe, are known analytically, but for larger atoms and for molecules, liquids, solids, etc., and even for relativistic modeling of the H atom, we almost always obtain eigenenvalues (energies) using numerical methods. For this exact reason, a chemist by the name of Ernest Davidson came up with one of the best ways to find the lowest eigenvalue of a matrix, and this is called the Davidson method. In only about 3.5 years, the word "eigenvalue" comes up 163 times on MMSE, so you can find a lot of real-world uses of eigenvalues.

$\endgroup$
5
$\begingroup$

In seismology, decomposing the eigenvalues is used to calculate the fault plane (and auxillary fault plane as this is ambiguous) of an earthquake. As movement is assumed only on one plane and there is no volumetric change, you:

  • take the strain tensor modelled from p-wave arrivals at a bunch of observation stations around the world,
  • minimise the Identity matrix (to minimise the volumetric displacement)
  • the resulting triangular matrix contains enough information to work out the plane of movement (eigenvectors) and the magnitude (eigenvalues).

Apologies for the lack of precision; I studied geophysics over 15 years ago.

Fun fact: if you struggle to minimise the Identity matrix from a seismic event then your "event" is likely to be an underground nuclear test, as nuclear explosions produces a large initial p-wave with volumetric displacement.

$\endgroup$
1
  • 1
    $\begingroup$ Would SVD not work for this? $\endgroup$ Dec 19, 2023 at 17:52
2
$\begingroup$

I gave a more detailed answer about dynamical systems in my other answer on MSE, but here is a very concrete and important example.

The neutron flux density in a reactor is governed by the neutron transport equation. Here is a simplified version of the steady-state equation: $$ (\Omega\cdot\nabla + \Sigma_t)\psi = \int\mathrm{d}E^{\prime} \int\mathrm{d}\Omega^{\prime} \Sigma_s \psi + \frac{1}{k} \chi \int\mathrm{d}E^{\prime} \nu \Sigma_f \int\mathrm{d}\Omega^{\prime} \psi. $$

Here $\psi$ and $k$ are unknown, so this is an eigenvalue problem. The largest value of $k$* determines the multiplicative factor governing the criticality of the reactor. If $k<1$, the reactor is not self-sustaining and will choke itself out. If $k=1$, the reactor is self-sustaining and provides stable power output. If $k>1$, you have an unstable reaction at best, and a bomb at worst.

Many factors in reactor design influence $k$, such as reactor geometry, material properties, moderator and coolant flow throughout the reactor, etc. Even though this equation is linear, it contains 6 independent variables ($E\in [0,\infty)$, $\Omega\in S^2$, $x\in V\subset\mathbb{R}^3$), which result in intractably large discretizations without significant simplifications and assumptions as well as proper algorithm selection for the solving the eigenvalue problem.

*: I am not familiar with analysis that establishes spectral properties of this problem, but this is generally accepted in practice.

$\endgroup$
0
$\begingroup$

lots in biology; for example, predator-prey systems. Wolves move to areas with high deer density; deer move to areas with few wolves. How trees grow branches, how small animals disperse seeds, how bacteria spread based on solution....

http://mitran-lab.amath.unc.edu/courses/MATH564/biblio/text/10.pdf

$\endgroup$
1
  • 5
    $\begingroup$ I ctrl-F'ed that document and couldn't find any mention of eigenvalues or eigenvectors. $\endgroup$ Dec 19, 2023 at 19:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.