I am looping an array using MPI. The problem is, i think that some processes are moving onto their next iteration before other precesses have finished theirs. This is causing me problems because data calculated by each process at each iteration is needed in the next. Is their a way that i can pause the processes at the end of the loop until all other have finished their current iteration? (Im using Fortran90 if it makes any difference) Thanks!
An MPI_Barrier can be used to synchronize all processes in a communicator. Each of the processes has to wait till all other processes reach the barrier before all of them can proceed further:
Here COMM is the communicator handle and IERROR the error status.
Note that sometimes there are more clever ways to deal with such dependencies algorithmically, but this is highly problem-dependent.
While Christian has given you the right answer, it's almost certainly not what you want. Unnecessary synchronization is guaranteed to limit the scalability of your code. If all processors need data from all other processors, then using a collective like MPI_Alltoall() might be more appropriate. If one task has received all the required data from other tasks, there's usually no harm in that task continuing even if others are still working on the previous iteration. In that case, you can probably use point-to-point communications only.