MPI in C, all processor run outside of the MPI envir created

Question

I am new to MPI and am trying to figure out the subtleties in it, and for that reason I am just playing around with an easy example. The question I have is that when I run the code below with 2 processors, even though the printf statement is BEFORE the MPI_Init both processors print it. Is there anyway to not have this happen, and only use the multiple processors when the environment is initialized? Below is my minimum working example.

#include <mpi.h>
#include<stdio.h>

#define send_data_tag 2001
#define return_data_tag 2002

double add(int a);

int main(int argc, char ** argv)
{
  int rank, size, len;
  char host[MPI_MAX_PROCESSOR_NAME];

  printf("Will all processors print this?");
  MPI_Status status;
  MPI_Init(&argc, &argv);

  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
  MPI_Comm_size(MPI_COMM_WORLD, &size);
  MPI_Get_processor_name(host, &len);

  if(rank == 0)
    {
        int test = 1;
        int final_test;
        MPI_Send(&test, 1 , MPI_INT, 1, send_data_tag, MPI_COMM_WORLD); 

        MPI_Recv(&final_test, 1, MPI_INT, 1, return_data_tag, MPI_COMM_WORLD, &status);
        printf("%d\n", final_test);
    }
  else
    {
        int test_recv , final_test;
        MPI_Recv( &test_recv, 1, MPI_INT, 0, send_data_tag, MPI_COMM_WORLD, &status);
        final_test = test_recv + 1;
        MPI_Send(&final_test, 1, MPI_INT, 0, return_data_tag, MPI_COMM_WORLD);
    }


MPI_Finalize();
}

Answer 1

I fear you're confusing your parallel paradigms. MPI supports parallelism with multiple processes (think instances of your program) which communicate with each other across a network. It's possible the "network" can be within one box, or it can be on multiple machines miles away from each other. With this in mind, it's easy to see why your print command is being run multiple times. Each separate process starts up, prints the string, then actually gets round to setting up communication with the other processes. Thus the only way to limit IO to one process is with conditionals like your if (rank==0).

Another parallel model, threading, works on a single box in a single process with threads sharing access to the same memory. This model fits better with the viewpoint you describe in which threads are only created when there is work to be done in parallel. If you're interested in this, then look up some tutorials on OpenMP, (not to be confused with openMPI, a well known implementation of MPI) which seems to be the popular way to thread a program with minimum effort at the moment.

Answer 2

Usually, in MPI programs, the MPI_Init(), MPI_Comm_rank and MPI_Comm_size are used only once in the very beginning. They initialize the environment, assign the "rank" and get the communicator size in the MPI_COMM_WORLD communicator (one may set multiple communicators for convenience in some scenarios). But your question is still interesting.

Compile and run this small code with MPI:

#include<stdio.h>
#include <unistd.h>

int main(int argc, char ** argv)
{
  printf("Will all processors print this?\n"
          "My process ID : %d\n"
          "My parent's ID: %d\n",
          getpid(),getppid());
}

This will print the ID of each process you start with the mpirun, as well as the parent process.

Notice that not even the mpi.h is included here. What you see is because you start several processes with mpirun. Each of them gets the same binary, but can follow a different branch because MPI assigns a different rank to it.

If you want to do something only with one process, you have to put it in an if (rank==0) (or a similar statement), as origimbo mentioned. Ofcourse, this needs to be after the MPI_Init() and the MPI_Comm_rank() calls.

Answer 3

If that is what you really want, there is MPI_COMM_SPAWN(...)( and the more general MPI_COMM_SPAWN_MULTIPLE(...)). It can be used to start and communicate with a second (separate) MPI program. You would have to split your code into two sections, the initial serial part and then the subsequent parallel part that will be started by MPI_COMM_SPAWN(...)