0
$\begingroup$

I have been learning petsc recently and made a simple solver for stokes equation on 3D staggered grid. My code is running on server with 200GB ram, but if I run a solver on 256^3 grid, memory used by the solver is over 200GB.

Checking with the -log_view flag, I've figured out most of the memories were used by the matrix created by 

DMCreateMatrix(Mat &A)

I understand that the DM structure helps the MPI communications and therefore may need extra memories, but it seems too much in my opinion. Is there any way to reduce the memory usage by Matrix created from DM objects?

Improve this question
$\endgroup$
3
  • $\begingroup$ That's a question you should probably ask on the PETSc mailing lists. But you don't say (i) how the matrix you build looks, for example how many entries it has per row, and (ii) how much memory the matrix actually requires? $\endgroup$ – Wolfgang Bangerth Mar 31 '20 at 13:34
  • $\begingroup$ The matrix size is about 256^3*4 by 256^3*4 with each row consists of a maximum of 9 non-zero entries. As far as I know, it implies that 256^3*4*9 of doubles are allocated, which costs less than 5 GB. Integer indexes are stored too, but it will still cost less than array of double. Therefore I expect that memory by the matrix must not exceed 20GB, and when the matrix is created without DM object, by using MatCreatAij, it is under 20GB. By the way, I better ask on the PETSc mailing lists. $\endgroup$ – Hoarsehinghing Apr 1 '20 at 2:04
  • $\begingroup$ I've found that it has something to do with stencil_width. DMSTAG_STENCIL_BOX with stencil width 2, makes 200GB usage, where as DMSTAG_STENCIL_STAR with stencil width 1 gives me a reasonable memory. $\endgroup$ – Hoarsehinghing Apr 1 '20 at 7:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.