I asked this question over at StackOverflow and someone told me that I'd get a better answer here. So here's my problem:

I'm working on a machine learning project which involves doing a Principal Component Analysis on some labeled data and using those labels to extract more valuable information from the data.

To do that, I'm calculating a scatter matrix for each class, and for each pair of classes I need to solve a generalised eigenvalue problem for their scatter matrices, as follows:

S_i * v = w * (S_j + b.I) * v

where b is a multiplier and I is the identity matrix. Now, this is the code in python:

        jeigenvalues = eigsh(scatter_j, k=10, return_eigenvectors=False, maxiter=100)
        print('eigenvalues made')
        beta = betaMult*mean(jeigenvalues)
        print(beta)
        print(scatter_j+beta*eye(shape(x_data)[1]))
        w, v = eigsh(scatter_i,M=scatter_j+beta*eye(shape(x_data)[1]),k=int(numberOfEVs/45), maxiter=100)
        print(i,j,'done')

numberOfEVs is 90 in my current code (so that it's divisible by 45).

But the problem is, at the line where I use the eigsh for the aforementioned formula, it never gives me an answer. It keeps eating more and more memory without even completing a single iteration (I set its maxiter input to 1, and it still didn't give an answer). When I don't give the eigsh function the M argument (which is the matrix on the right side of the generalised EV problem and it is assumed to be "I" when not specified), it works correctly. But when M is provided, it becomes unresponsive.

Any ideas?

EDIT: The scatter matrices have rather small entries, mostly around 10^-5. I've also tried multiplying the left hand side by the inverse of the RHS matrix, and again it's having the same issue (goes on for a long time without an answer). Is the smallness of these entries the issue? How can I solve it, then?

I asked this question over at StackOverflow and someone told me that I'd get a better answer here. So here's my problem:

I'm working on a machine learning project which involves doing a Principal Component Analysis on some labeled data and using those labels to extract more valuable information from the data.

To do that, I'm calculating a scatter matrix for each class, and for each pair of classes I need to solve a generalised eigenvalue problem for their scatter matrices, as follows:

$$S_i v = w (S_j + \beta I) v,$$

where $\beta$ is a multiplier and $I$ is the identity matrix. Now, this is the code in python:

        jeigenvalues = eigsh(scatter_j, k=10, return_eigenvectors=False, maxiter=100)
        print('eigenvalues made')
        beta = betaMult*mean(jeigenvalues)
        print(beta)
        print(scatter_j+beta*eye(shape(x_data)[1]))
        w, v = eigsh(scatter_i,M=scatter_j+beta*eye(shape(x_data)[1]),k=int(numberOfEVs/45), maxiter=100)
        print(i,j,'done')

numberOfEVs is 90 in my current code (so that it's divisible by 45).

But the problem is, at the line where I use the eigsh for the aforementioned formula, it never gives me an answer. It keeps eating more and more memory without even completing a single iteration (I set its maxiter input to 1, and it still didn't give an answer). When I don't give the eigsh function the M argument (which is the matrix on the right side of the generalised EV problem and it is assumed to be "I" when not specified), it works correctly. But when M is provided, it becomes unresponsive.

Any ideas?

EDIT: The scatter matrices have rather small entries, mostly around 10^-5. I've also tried multiplying the left hand side by the inverse of the RHS matrix, and again it's having the same issue (goes on for a long time without an answer). Is the smallness of these entries the issue? How can I solve it, then?