This might be a silly quesntion but recently I've been trying to program the eigenface algorithm using PCA, so I arranged the face vectors vertically in a matrix X such as: X = [x1,x2,x3,...,xn]; In this case, what would be the wright way of computing the covariance matrix? cov(X) or cov(X')?

  • $\begingroup$ Depends on how the covariance function is implemented in your specific programming environment, but with data along the columns you should have $\Sigma = \frac{1}{n-1}(X-\mu\mathbf{1}^T)(X-\mu\mathbf{1}^T)^T$ if you want to do it yourself $\endgroup$ – whpowell96 Oct 3 at 2:18
  • $\begingroup$ Thanks for the comment, but one thing I have trouble understanding is why isn't the covariance in this case the inner product of X with itself, is there any intuitive explanation? $\endgroup$ – Marcus Oct 3 at 2:53
  • 2
    $\begingroup$ Note that to compute the basis you should not form the covariance matrix, but instead compute the SVD of X (with or without removing its average first, it doesn’t matter much in practice). Then the columns of U are your basis. Computing the product X^T X directly is bound to introduce round-off errors, which tend to corrupt the basis vectors, especially the ones associated with the smaller singular values. $\endgroup$ – Amit Hochman Oct 3 at 3:51
  • $\begingroup$ Do you have any sugestion of books I could read on the issue? $\endgroup$ – Marcus Oct 3 at 11:15
  • 1
    $\begingroup$ Also for a more heuristic argument, if we had a matrix of inner products, the resulting matrix would be size $n\times n$, but $n$ has nothing to do with the underlying random process, it is just the number of realizations we have access too. Any covariance matrix would have to be size $m\times m$ where $m$ is the dimension of each data point $x_i$. $\endgroup$ – whpowell96 Oct 3 at 19:38

I believe that eigen faces need a non-linear version of PCA. See two examples here

https://science.sciencemag.org/content/290/5500/2323 https://science.sciencemag.org/content/290/5500/2319

for some of the seminal papers that implement such non-linear PCA approaches on faces.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.