# How do I compile a program that contains both MPI and OPENMP

I have a fortran 90 code that distributes blocks of computations (from a matrix) to multiple nodes in a cluster using MPI, but in each node, the for loops are executed in parallel using openmp. I tried using the following command:

mpif90 mycode.f


which produces the following output:

mycode.f(49): (col. 3) remark: LOOP WAS VECTORIZED.
mycode.f(68): (col. 4) remark: PARTIAL LOOP WAS VECTORIZED.
mycode.f(83): (col. 2) remark: LOOP WAS VECTORIZED.
mycode.f(92): (col. 2) remark: LOOP WAS VECTORIZED.
mycode.f(105): (col. 2) remark: LOOP WAS VECTORIZED.
mycode.f(114): (col. 2) remark: LOOP WAS VECTORIZED.
/tmp/ifortAPx6mR.o: In function MAIN__':


I assume that the problem is that I need to set a parameter called omp_set_num_threads_ to a value somewhere, but I'm not sure where. Any help would be greatly appreciated. Thanks.

• Is this on linux? If so, you might want to set the environment variable OMP_NUM_THREADS.
– Dan
Jan 12 '12 at 18:33
• @Dan - OMP_NUM_THREADS won't matter at compile time.
– user389
Jan 12 '12 at 19:59
• I misread the question. Thanks for pointing that out.
– Dan
Jan 12 '12 at 20:02
• You appear to be using ifort. Adding the flag to enable OpenMP support is required. The flag is "-openmp". See "ifort -help" for more information. Mar 17 '13 at 1:35

mpif90 is just a wrapper for your underlying fortran compiler, which adds the necessary options for finding the necessary libraries and/or include files/modules.

You just need to add the options necessary for compiling/linking with OpenMP; that would be -fopenmp (for the gnu compiler suite) or -openmp (for intel); not sure about others. "remark: LOOP WAS VECTORIZED" looks like the intel compilers to me, so I'd suggest just trying

mpif90 mycode.f -openmp

• Excellent! Now that it compiles, how would I go about executing the file?
– Paul
Jan 16 '12 at 3:14
• You would just mpirun the program as usual, if you're ok with the default OMP_NUM_THREADS (generally, the number of cores you ahve) being the number of threads per task; if not, you'll want to set OMP_NUM_THREADS to equal the desired value in each of those tasks. Depending on what your MPI exports, you could just try export OMP_NUM_THREADS=<whatever>; mpirun -np <otherwhatever> ./mycode.
– user389
Jan 16 '12 at 3:20

Totally normally, with OpenMP enabled in the compiler and you including/linking MPI headers/libraries. Then when you start your job with mpiexec (or equivalent) the parts of your code written using MPI will run as MPI, parts marked with OpenMP pragmas (automatically or not) will run as OpenMP. There is nothing special in compilation, the problem is to write efficent code.