# Eigenfaces Algorithm

This might be a silly quesntion but recently I've been trying to program the eigenface algorithm using PCA, so I arranged the face vectors vertically in a matrix X such as: X = [x1,x2,x3,...,xn]; In this case, what would be the wright way of computing the covariance matrix? cov(X) or cov(X')?

• Depends on how the covariance function is implemented in your specific programming environment, but with data along the columns you should have $\Sigma = \frac{1}{n-1}(X-\mu\mathbf{1}^T)(X-\mu\mathbf{1}^T)^T$ if you want to do it yourself – whpowell96 Oct 3 at 2:18
• Thanks for the comment, but one thing I have trouble understanding is why isn't the covariance in this case the inner product of X with itself, is there any intuitive explanation? – Marcus Oct 3 at 2:53
• Note that to compute the basis you should not form the covariance matrix, but instead compute the SVD of X (with or without removing its average first, it doesn’t matter much in practice). Then the columns of U are your basis. Computing the product X^T X directly is bound to introduce round-off errors, which tend to corrupt the basis vectors, especially the ones associated with the smaller singular values. – Amit Hochman Oct 3 at 3:51
• Do you have any sugestion of books I could read on the issue? – Marcus Oct 3 at 11:15
• Also for a more heuristic argument, if we had a matrix of inner products, the resulting matrix would be size $n\times n$, but $n$ has nothing to do with the underlying random process, it is just the number of realizations we have access too. Any covariance matrix would have to be size $m\times m$ where $m$ is the dimension of each data point $x_i$. – whpowell96 Oct 3 at 19:38