The AVIDD-O Cluster
MPICH-GM Guide
Versions of MPICH-GM (MPI over Myrinet) have been compiled with both the PGI Compilers and GCC. The respective Softenv keys are:- +mpich-gm-pgi
- +mpich-gm-gcc
Please note that these are pointers to the actual Softenv keywords. These keywords will always point to the latest versions of MPICH-GM compiled with the latest versions of GCC and the Intel Compiler.
Building software with MPICH-GM
Building MPICH-GM-enabled code is as easy as running:mpicc -o my_parallel_binary my_parallel_source.c
although it can be quite a bit more complicated, as well. Compiler and library paths should be dealt with automatically via Softenv.
If you have difficulties with mpicc or Softenv, please see the respective man pages or send mail to the High Performance Systems group, hps-admin@iu.edu.
Using mpirun
MPICH-GM-enabled binaries are executed with mpirun, which handles the spawning of processes on multiple nodes. This simple example, which is based on node-allocation provided by PBS, would run 8 copies of the binary para_code on four different AVIDD compute nodes (after running qsub -l nodes=4:ppn=2 para.pbs, for example):mpirun -machinefile $PBS_NODEFILE -np 8 ~/para_code
mpirun should only be used from within a PBS job script. Defining your own "machinefile" and executing mpirun from an arbitrary node will almost certainly result in poor performance, since those nodes that you choose will likely have already been allocated to another user by PBS. Administrators will terminate any processes found on compute nodes that are not under the control of PBS.
Further reading
See $MPICH_GM_HOME/doc for more documentation on MPICH-GM.
