I am encountering a problem that my MATLAB code ran and an error occurs and shows that it is out of memory. My code is using finite element method to solve some complicated kind of integral equation and the number of elements can reach more than 100,000. I use the sparse to store the stiffness matrix and ran the code in parallel on our school cluster. I am not sure whether the data is too big or my code is not well programmed that caused the out-of-memory problem. I also have the c version of the code and same problem occured(the error showed to be segmentaion fault). For small data my code does not have anything wrong. Since it is a 2d FEM code, suppose the number of unknown in each coordinate is n(so that the total is n^2), then the bandwidth of the matrix is approximately less than 2*n. It is symmetric positive definite. When I use MATLAB, I just use \ to solve the linear system. For c version, I use the subroutine for banded matrix in CLAPACK. It is the big data problem and my code does not work, so is there anybody who would like to help me and tell me what I should do to fix it, no matter in c or MATLAB? And for c, is there any package available for solving linear system with big data? Thanks so much!
While Bill and tbirdal's suggestions of looking at other sparse solvers are good, I suggest trying Matlab's own sparse iterative methods first, since you already have something working. From looking at matlab's documentation on the A\b operator, it's always going to use a direct decomposition method, which as Bill said, 100k unknowns might be too big for. Since A is symmetric positive definite, try the preconditioned conjugate gradient method -- x = pcg(A,b).
Take a look at Intel MKL or ACML. Especially MKL provides efficient and parallel sparse solvers on CPU. Both direct and indirect ones are present. As a second alternative, also look at PARDISO solver here.