I asked this question over at StackOverflow and someone told me that I'd get a better answer here. So here's my problem:
I'm working on a machine learning project which involves doing a Principal Component Analysis on some labeled data and using those labels to extract more valuable information from the data.
To do that, I'm calculating a scatter matrix for each class, and for each pair of classes I need to solve a generalised eigenvalue problem for their scatter matrices, as follows:
$$S_i v = w (S_j + \beta I) v,$$
where $\beta$ is a multiplier and $I$ is the identity matrix. Now, this is the code in python:
jeigenvalues = eigsh(scatter_j, k=10, return_eigenvectors=False, maxiter=100) print('eigenvalues made') beta = betaMult*mean(jeigenvalues) print(beta) print(scatter_j+beta*eye(shape(x_data))) w, v = eigsh(scatter_i,M=scatter_j+beta*eye(shape(x_data)),k=int(numberOfEVs/45), maxiter=100) print(i,j,'done')
numberOfEVs is 90 in my current code (so that it's divisible by 45).
But the problem is, at the line where I use the
eigsh for the aforementioned formula, it never gives me an answer. It keeps eating more and more memory without even completing a single iteration (I set its
maxiter input to 1, and it still didn't give an answer). When I don't give the
eigsh function the
M argument (which is the matrix on the right side of the generalised EV problem and it is assumed to be "I" when not specified), it works correctly. But when
M is provided, it becomes unresponsive.
EDIT: The scatter matrices have rather small entries, mostly around 10^-5. I've also tried multiplying the left hand side by the inverse of the RHS matrix, and again it's having the same issue (goes on for a long time without an answer). Is the smallness of these entries the issue? How can I solve it, then?