Manipulating a generalized eigenvector problem to plain eigenvector problem

Question

Let $X\in\mathbb{R}^{n\times p}$ denote a matrix with $p$ linearly-independent columns, and let $L\in\mathbb{R}^{n\times n}$ denote a symmetric matrix. Furthermore, let $D\in\mathbb{R}^{n\times n}$ denote a diagonal matrix with positive-entries.

Now suppose that we are interested in computing solutions to the symmetric generalized-definite eigenproblem,

$$ A v = \lambda B v, $$

where the symmetric matrices $A$ and $B$ (where $B$ is also positive-definite) are implicitly given by the matrices $X$, $L$, and $D$ above, where $A = X^T L X$, and $B = X^T D X$.

I ran into a paper which claimed that we may solve an equivalent standard eigenvalue problem involving a modification of $X$. In particular, if we orthogonalize $X$ with respect to $D$, say $\tilde X^T D \tilde X = I$, then eigenpairs of

$$ \tilde X^T L \tilde X u = \lambda u $$

are also eigenpairs for the original generalized eigenproblem. How can we show this to be true?

Answer 1

Let $X=\tilde X Z$, where $\tilde X^T D \tilde X=I$. Then we may rewrite

$$ (X^T L X)v = \lambda (X^T D X) v $$

as

$$ (Z^T \tilde X^T L \tilde X Z) v = \lambda (Z^T Z)v. $$

Now, put $u=Z v$, and premultiply both sides of the previous equation by $Z^{-T}$ to find

$$ \tilde X^T L \tilde X u = \lambda u. $$

It is worth noting that, contrary to your last sentence, the $u$ and $v$ are not the same, but are related through the equation $u=Z v$. Thus, after solving the standard eigenvalue problem with $\tilde X^T L \tilde X$, linear systems will need to be solved with $Z$ in order to find the original eigenvectors.

It is also important to notice that the fact that $D$ is positive is crucial to the ability to orthonormalize $X$ with respect to $D$. Consider the case where $D=-I$. Then requiring that $\tilde X^T D \tilde X = I$ is equivalent to requiring that $\tilde X^T \tilde X = -I$, which is not possible for real $\tilde X$ (nor in the complex case, when we replace transposes with Hermitian-transposes). When $D$ is positive, we may define

$$ Z = \sqrt{X^T D X}, $$

which will exist because $X^T D X$ will be positive-definite.