What is the policy of multiple overlapping asynchronous transfers in MPI?
I have a program with several open asynchronous
irecv operations. I find that transfers that could take place (the corresponding
isend has been called) wait on other transfers that aren't yet ready (the corresponding
isend hasn't yet been called). To be clear this inefficiency does not stem from network contention; my network is unnecessarily idle.
My program looks like the following:
call irecv(variable A from machine 2) call irecv(variable B from machine 2) call irecv(variable C from machine 2) call wait(variable C from machine 2) call do_important_work_with(variable C) ....
call isend(variable C to machine 1) call isend(variable B to machine 1) call do a bunch of costly work call isend(variable A to machine 1) ....
The transfer of
C seems to be unnecessarily blocked by the transfer of
I find that the
variable C on Machine 1 does not complete until after the costly work on Machine 2 completes. This is unfortunate because this transfer could have started at the beginning of my program. It seems to wait unnecessarily for the transfer of
A to complete.
In particular I have a computation like the following.
- Is this expected?
- What is the policy of multiple overlapping asynchronous transfers?
- Can this be avoided without rearranging my code (is there some relevant internal setting)?
- Where should I go to learn more about MPI's policy for multiple live transfers?