8
$\begingroup$

In MPI, is there any built-in mechanism to notify a group of processes that they need to receive messages from other processes?

In my application every process needs to send data to a group of processes with known rank IDs (which potentially changes at each iteration), but they do not know the rank of the processes that will be receiving messages from. Is there a portable, preferably built-in, way of achieving this without querying each and every process?

Improve this question
$\endgroup$

4 Answers 4

Reset to default
7
$\begingroup$

You could try having all processors use MPI_IProbe or MPI_Probe with MPI_ANY_SOURCE to check if there are any receivable messages with a given tag. If there are matching messages, you can extract the senders rank from the returned status and call MPI_Recv immediately.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Is calling MPI_Probe essentially cheaper than MPI_Recv ? $\endgroup$
    – mmirzadeh
    Jul 5, 2013 at 22:26
  • $\begingroup$ @GradGuy With MPI_IProbe and MPI_Probe you don't actually receive any messages (you are just querying MPI to see if there are any receivable messages), so in that sense I suppose it is cheaper than MPI_Recv. $\endgroup$
    – Matthew Emmett
    Jul 7, 2013 at 3:54
  • $\begingroup$ @MatthewEmmett How receiver knows how many MPI_IProbe calls to post or how long keep probing? $\endgroup$
    – Shibli
    May 21, 2017 at 14:54
12
$\begingroup$

Others have already proposed the various MPI_Probe variants but I'd like to point out one thing: MPI is not a remote procedure call, i.e., there are no ways to notify a process that some message has come in (e.g., by raising a signal). Messages are sent but if the receiving process doesn't actually go look for them, then nothing will happen. As such, the answer to your question is "no", but a process can actively figure out whether a message has been sent to it.

Improve this answer
$\endgroup$
5
  • $\begingroup$ In theory, MPI_Probe can be descheduled by the kernel such that it is passive until used, which is not too different from how IBM implements real RPCs on Blue Gene (where the OS and HW collaborate to implement very efficient network-initiated interrupts that can call arbitrary user-defined callbacks). Of course, no implementation I know about actually does this with blocking MPI calls... $\endgroup$
    – Jeff Hammond
    Jul 6, 2013 at 20:06
  • 1
    $\begingroup$ The point I wanted to make is not so much about how it's actually implemented, but about the mental model one should have when thinking about MPI. MPI is purely a passive concept: you can't access data from another process, you can't expect a signal to be raised if an incoming message arrives; everything only ever happens if you actively send a message or look whether one came in. $\endgroup$
    – Wolfgang Bangerth
    Jul 7, 2013 at 18:46
  • $\begingroup$ You're right about a signal but RMA gives you access to remote data and implementations are supposed to provide passive progress. $\endgroup$
    – Jeff Hammond
    Jul 10, 2013 at 19:58
  • 1
    $\begingroup$ Yes, you are talking about what MPI 3 calls "one-sided communication". That's a fairly recent addition, and likely a rarely used one. But I think it doesn't take away from the mental model I laid out above. $\endgroup$
    – Wolfgang Bangerth
    Jul 11, 2013 at 11:39
  • $\begingroup$ MPI 2.0 had RMA ten years ago and one can implement e.g. put_with_notify using those features. I agree that MPI does not have remote procedure calls aka active messages (yet) but no non-IBM machine implements those without polling, so we're not really talking about MPI any more if we're criticizing the need for polling. $\endgroup$
    – Jeff Hammond
    Jul 14, 2013 at 19:13
7
$\begingroup$

http://mpitutorial.com/tutorials/dynamic-receiving-with-mpi-probe-and-mpi-status/ has a tutorial describing the use of MPI_Probe that might be useful to you.

Improve this answer
$\endgroup$
1
$\begingroup$

If the senders know to whom they are sending but the receivers don't know from whom they are receiving, just post your receives from MPI_ANY_SOURCE. That will match the receive with any process that sends a message to it.

If the receivers might not actually receive anything at all, they can post an iRecv. Whenever a process is done sending its messages, it can post am Ibarrier. When the Ibarrier completes because all processes have entered it (and called MPI_WAIT on the request object), everyone knows that the communication is done and they can cancel their previous Irecv.

Improve this answer
$\endgroup$
6
  • $\begingroup$ In what order does that receive the message? I need to send back the senders result of some calculation ... $\endgroup$
    – mmirzadeh
    Jul 3, 2013 at 23:57
  • $\begingroup$ The receives will be in order per sending process. If rank 1 sends messages A - B - C in that order, they will arrive in that order. There's no guarantee of the ordering between the processes. $\endgroup$
    – Wesley Bland
    Jul 3, 2013 at 23:58
  • $\begingroup$ I see ... but is it guaranteed that the messages from processes are not mixed together? $\endgroup$
    – mmirzadeh
    Jul 3, 2013 at 23:59
  • $\begingroup$ No. The messages between processes can arrive in any order. $\endgroup$
    – Wesley Bland
    Jul 4, 2013 at 0:38
  • 3
    $\begingroup$ MPI_Cancel is a horrible function and should be used as infrequently as possible. I highly endorse variants of Probe instead. $\endgroup$
    – Jeff Hammond
    Jul 4, 2013 at 1:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.