I was trying to understand more in detail how a blocking MPI all to all really work. Of course, the actual all to all traffic is pretty straightforward, what I can't understand is how it is able to "keep" blocking until the collective is done.
I currently have two theories:
- For each rank the call is blocking until it has fully received the data from all its posted receives. But this seems counterintuitive to me as not all ranks will release at the same time then.
- Each rank keep track of all the other ranks and it only mark the collective as finished when all other ranks signals to it (I guess with some sort of broadcast) that they have received all the expected data.
Note that I am a noob to MPI and I could be completely wrong, so feel free to school me or point to me to a clear resource where it is explained.