A typical way of dealing with I/O in MPI parallel programs is to either read all data to a single node and dispatch to the other nodes accordingly, or send all data to a single node and write from this node.
I currently use blocking communication.
Since the "master" process can only communicate with one node at a time, most nodes are stuck in the blocking communication step : they essentially "do nothing" while they wait for the master to send/receive. This is completely understandable.
However, when I have a look at CPU usage using "top", all copies of the program have a full 99 - 100% CPU usage. This means that (contrary to my initial guess) waiting for the other side of the communication is not an "idle task" taking up a fairly low amount of CPU, but rather a "busy" one.
How can I reduce the CPU usage for these "waiting" nodes ? Is non-blocking communication relevant here ?
Thanks for any answer,