Problems running a PETSc example in parallel

Question

After configuring and building PETSc, I have successfully been able to run several examples. In particular, I am working with this example. I have been able to run the program using the following commands:

make ex2 mpiexec -n 4 ./ex2 -m 40 -n 40

which produces the following output:

Norm of error 0.000642883 iterations 26
Norm of error 0.000642883 iterations 26
Norm of error 0.000642883 iterations 26
Norm of error 0.000642883 iterations 26 

This seems to tell me that the same problem was solved 4 times, rather than once by four processors in parallel. Suspicious, I ran the program again using

mpiexec -n 4 ./ex2 -m 40 -n 40 -log_summary

which produced the following output (note that it says that ./ex2 was run with only 1 processor:

Norm of error 0.000642883 iterations 26
************************************************************************************************************************
***             WIDEN YOUR WINDOW TO 120 CHARACTERS.  Use 'enscript -r -fCourier9' to print this document            ***
************************************************************************************************************************

---------------------------------------------- PETSc Performance Summary: ----------------------------------------------

./ex2 on a petsc-arc named utepgeon01.utep.edu with 1 processor, by pmdelgado2 Tue Jan 24 22:16:23 2012
Using Petsc Release Version 3.2.0, Patch 6, Wed Jan 11 09:28:45 CST 2012 

                         Max       Max/Min        Avg      Total 
Time (sec):           6.203e-02      1.00000   6.203e-02
Objects:              4.500e+01      1.00000   4.500e+01
Flops:                3.230e+06      1.00000   3.230e+06  3.230e+06
Flops/sec:            5.207e+07      1.00000   5.207e+07  5.207e+07
Memory:               7.996e+05      1.00000              7.996e+05
MPI Messages:         0.000e+00      0.00000   0.000e+00  0.000e+00
MPI Message Lengths:  0.000e+00      0.00000   0.000e+00  0.000e+00
MPI Reductions:       4.730e+02      1.00000

I know that the computer I'm using has 4 processors (after typing grep processor /proc/cpuinfo). After checking my reconfigure-petsc-arch.py file, I see the following:

#!/usr/bin/python
if __name__ == '__main__':
  import sys
  import os
  sys.path.insert(0, os.path.abspath('config'))
  import configure
  configure_options = [
    '--download-f-blas-lapack=1',
    '--download-mpich=1',
    '--with-cc=gcc',
    '--with-fc=gfortran',
    'PETSC_ARCH=petsc-arch',
  ]
  configure.petsc_configure(configure_options)

Is there something else I need to do to distribute the work of solving the linear system in parallel?

Answer 1

After consulting the resources suggested, I found out that the computer that I was working with already had a different version of MPI installed on it, which PETSc was trying to use everytime I called the 'mpiexec' commmand. When I built PETSc, I asked it to install MPI as well. So, whenever I want to run a PETSc program in parallel, I needed to use the version that was installed with PETSc, not the one that was installed previously.

To do this, I looked in my $PETSC_DIR/$PETSC_ARCH/bin and found mpiexec in this directory. So, I could run PETSc in parallel if I called it from the command prompt as:

$PETSC_DIR/$PETSC_ARCH/bin/mpiexec -n 4 ./ex2 -m 40 -n 40 -log_summary

where $PETSC_DIR is the main directory where all the PETSc files were installed, and $PETSC_ARCH is the directory where all the PETSc architecture configuration files are installed. However, to avoid having to type so much at the command prompt, I edited my .bashrc file (in my home directory in linux) with the following alias:

alias petsc="~/petsc-3.2-p6/petsch-arch/bin/mpiexec"  

(note: the ~ is there because I have it installed in my home directory) This way, anytime I want to run a petsc file in parallel I could type something like this:

petsc -n 4 ./ex2 -m 40 -n 40 -log_summary