18
$\begingroup$

I am parallelizing code to numerically solve a 5 Dimensional population balance model. Currently I have a very good MPICH2 parallelized code in FORTRAN but as we increase parameter values the arrays become too large to run in distributed memory mode.

I have access to a cluster with 15 nodes, where each node has two 8 core CPUs and 128GB of RAM. I want to write run a program with MPI-3.0 in shared memory mode so that each process does not generate its own copy of each array.

Before I can run anything on the cluster I have to test it on a desktop running Ubuntu. It is essentially one blade of the cluster in that it has two 8 core CPUs and 128 GBs of RAM. I will be writing and testing my code on it, so please gear your responses towards running programs on the Ubuntu computer.

I have read that there is a way to run MPI-3.0 in shared memory mode like OpenMP instead of its default distributed memory mode.

Questions:

  1. How will I have to alter my code? Do I need to add in calls to other MPI functions like MPI_WIN_ALLOCATE?

  2. How do I compile my code to run MPI-3.0 in shared memory mode? Will this be different if it is over several nodes?

Please give sample compilation scripts if you can. I also have only GNU compilers. The cluster I use does not support Intel compilers.

$\endgroup$
8
  • 1
    $\begingroup$ Yes you should be able to run MPI on a shared memory system. However, the way that it is actually programmed will be identical. Your code should be the same for both shared and distributed memory systems. How it is run is the only difference. $\endgroup$ Aug 16, 2015 at 18:14
  • $\begingroup$ Furthermore, although I have not tested this, I believe that you should be able to run your code using the command mpiexec -n 8 /path/to/application to fake your computer into thinking it has 8 different nodes. $\endgroup$ Aug 16, 2015 at 18:17
  • $\begingroup$ I have already run it on a shared memory system but I want the computing cores to share RAM, like in OpenMP. Some of my arrays are 6 GB so I need all the cores on each node to operate in shared memory mode. $\endgroup$ Aug 17, 2015 at 15:29
  • 2
    $\begingroup$ Well this is for a research project. So I need to scale the problem size up. I already have working code so the only thing preventing scale up is the memory needed. I have come across some documents and powerpoints from MPI conferences which suggest MPI-3.0 can share memory with new call funtions that have been added under the one-sided communication catagory. Functions like MPI_WIN_ALLOCATE_SHARE $\endgroup$ Aug 17, 2015 at 16:21
  • 2
    $\begingroup$ www.eurompi2014.org/tutorials/hoefler-advanced-mpi-eurompi14.pdf cs.utexas.edu/users/flame/BLISRetreat2014/slides/… $\endgroup$ Aug 17, 2015 at 16:31

1 Answer 1

1
$\begingroup$

I also found this link on Stack Overflow. I swear I looked for questions like mine for forever but it seems the best way to search a question on stack overflow is to start to make a question and it will suggest similar posts.

Anyway here is the link which does in fact say that you can run MPI3.0+ in shared memory mode like openmp. There are other sources I found which also suggest and some that state you can but do not explain how or give resources to figure it out.

https://stackoverflow.com/questions/24797298/mpi-fortran-code-how-to-share-data-on-node-via-openmp

After following the advice given here in this link I ran into issues with putting USE 'mpi_f08' at the top of my code. I am using gfortran-4.8 which apparently does not currently have enough FORTRAN2008 support to USE mpi_f08. There person in this link probably was using the intel fortran 2008 compiler. You can use USE mpi which covers up to Fortran2003 pretty well. However I do not think that Fortran 2003 has the Type() functions which are needed to use the Remote Memory Access functions, such as MPI_WIN_ALLOCATE_SHARED, so you cannot use them without FORTRAN2008. But I really do not have enough resources to definitely say that for a fact but what I have found does suggest that.

$\endgroup$
2
  • 1
    $\begingroup$ You can use MPI_Win_allocate_shared with all three of the Fortran interfaces. Furthermore, you can write your own with Fortran 2003 ISO_C_BINDING and the C interface. $\endgroup$ Aug 29, 2015 at 22:49
  • $\begingroup$ Please tell me why one needs Type() to use RMA from Fortran? I know of no such limitation. RMA has been used with F77 for many years. $\endgroup$ Aug 29, 2015 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.