For model reduction, I want to compute the left singular vectors associated to the - say 20 - largest singular values of a matrix $A \in \mathbb R^{N,k}$, where $N\approx 10^6$ and $k\approx 10^3$. Unfortunately, my matrix $A$ will be dense without any structure.
If I just call the svd
routine from the numpy.linalg
module in Python for a random matrix of this size, I run into a memory error. This is due to the allocation of $V\in \mathbb R^{N,N}$ for the decomposition $A = VSU$.
Are there algorithms around, that avoid this pitfall? E.g. by setting up only the singular vectors assiociated with nonzero singular values.
I am ready to trade in computation time and accuracy.
full_matrices
that be set to False so that only the 'nonzero' parts are computed. Nevertheless, is there a way to reduce the computation even further? $\endgroup$numpy
backend uses fortran code, theLAPACKE_dgesvd
routine for standard svd. However, typically your matrix isC_CONTIGOUS
(check withmatrix.flags
). Therefore it copies the data for fortran alignment. Additionally while running the lapack routine dgesvd another copy of your matrix is needed (or at least the memory for it). You can get rid of one copy if you make sure that the memory alignment is fortran style right from the beginning. $\endgroup$