In FEM applications, each processor needs only data from neighboring processors, which gives a stencil or star pattern of communication. In MPI_Win for MPI_Get, one needs to setup a group of processors to window on common data. But what we usually need is specifying a stencil pattern for MPI_Get. And it seems to me MPI_Win is a lot of waste, isn't it? Maybe there is a related question.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I'm not completely clear on what you're asking here, but it looks like a solution to your problem might be the new Neighborhood Collective in MPI-3.0 Chapter 7.6 (you can find the PDF here).
I'm far from an expert in these things, but the basic idea is that you attach a topology to your MPI_Communicator and get a new communicator where you can use sparse collective communications that don't have to involve all processes.
To use this, you'll have to make sure you have an implementation that supports the new features of MPI. I know that the latest version of MPICH (3.0.4) supports them. Last I heard, Open MPI hadn't finished their implementation yet, but I'm sure it's coming.
$\begingroup$
$\endgroup$
2
Wes is exactly right about neighborhood collectives, which had your usage (to the extent I understand it) as one of its motivators. However, like MPI RMA (i.e. MPI_Win), this is a static model that requires collective initialization. You could also use MPI_Probe and dynamically communicator between your neighbors. This may not allow for the same optimizations as neighborhood collectives or one-sided but it is dynamic and should work pretty well in any implementation of MPI-1.x forward (I do not recall when Probe was added).
-
$\begingroup$ Jeff, I'm not sure what your "collective initialization" is about, but one could use MPI_Ineighbor_allgather to combine stencil patterns and non-blocking completion. $\endgroup$ May 17, 2021 at 14:46
-
$\begingroup$ MPI windows and communicator creation are blocking collectives. $\endgroup$ May 19, 2021 at 19:14