Applications of Moore - Penrose generalized inverse of a matrix and associated projection?

Question

I am seeking applications in the industry for the Moore-Penrose generalized inverse $A^\dagger$ of a matrix $A$.

The Moore-Penrose Inverse of $A\in \mathbb{C}^{m\times n}$, denoted by $A^\dagger$, is the unique matrix $X$ satisfying the following four Penrose equations. \begin{eqnarray} (i)~ AXA = A,~ (ii)~ XAX = X, ~ (iii)~ (AX)^* = AX,~ (iv)~ (XA)^* = XA \end{eqnarray}

where $A^*$ denotes the conjugate transpose of a metrix. Associated with $A^\dagger$ we can define orthogonal projection operators $AA^\dagger$ and $AA^\dagger$.

Any references would be welcome.

Answer 1

Some uses of the pseudoinverse are in http://www.siam.org/search/?type=1&terms=pseudoinverse&search.x=0&search.y=0 .

But one can use the singular-value decomposition (SVD) whenever one can use the pseudoinverse, and the SVD is much more flexible to apply, and therefore much more used. Moreover, the pseudoinverse is numerically unstable and must be regularized in practice, and this is usually done via the SVD.

The SVD is ubiquitous in applications, too numerous to even give a sensible partial list.

Answer 2

The standard references for learning about generalized inverses (not just Moore-Penrose generalized inverses) are Generalized Inverses: Theory and Applications by Ben-Israel and Greville and Generalized Inverses of Linear Transformations by Campbell and Meyer.

One typically uses a QR decomposition (such as the $XY$ representation of projectors in "On the Numerical Analysis of Oblique Projectors", G. W. Stewart, SIMAX, 2011) or an SVD to calculate a pair of generalized inverses in practice, since the methods suggested in the textbooks above (normally some variant of LU decomposition) are, as Professor Neumaier points out, numerically unstable. In contrast, the QR- and SVD-based methods are far more stable.

Although not typically identified as such, generalized inverses play a role in any method identified as "Petrov-Galerkin" (or "Galerkin"). The orthogonality relation between the two relevant subspaces (these end up being the range and nullspace of a projector) is induced by a bilinear form; the two orthogonal subspaces determine a (nonunique) pair of generalized inverses. Such methods play a role in the numerical solution of partial differential equations, numerical sparse linear algebra (which you can see in the popular textbook Iterative Methods for Sparse Linear Systems by Yousef Saad, and model (order) reduction methods.

Ben-Israel and Greville discuss applications of generalized inverses such as:

• integral solutions of linear equations
• linear programmming
• least-squares solutions of inconsistent linear systems
• Tikohonov Regularization
• Difference equations
• and others...

Answer 3

Another application of the pseudoinverse is in preconditioning saddle point problems. Given a saddle point matrix

$$A = \begin{pmatrix} F & B^T \\ B & 0 \end{pmatrix},$$

arising in incompressible flow, Elman's "BFBt" preconditioner approximates the inverse of the Schur complement

$$S = B F^{-1} B^T$$

with

$$S_{BFBt}^{-1} = (BB^T)^{-1} B F B^T (BB^T)^{-1} .$$

This is essentially application of the Moore-Penrose pseudoinverse considering the identities (when $BB^T$ is nonsingular)

$$ \begin{align} B^\dagger &= B^T (BB^T)^{-1} \\ (B^T)^\dagger &= (BB^T)^{-1} B .\end{align}$$

This preconditioner was subsequently improved by moving the least squares argument from the discrete to the continuous setting, resulting in new diagonal scaling terms that significantly improve performance, see Elman et al (2006), Block preconditioners based on approximate commutators.