G4MPI is a native interface with MPI libraries. The directory contains a Geant4 UI library and a couple of parallelized examples. Using this interface, users applications can be parallelized with different MPI compliant libraries, such as OpenMPI, LAM/MPI, MPICH2 and so on.

System Requirements:

MPI Library

The MPI interface can work with MPI-compliant libraries, such as Open MPI, LAM/MPI, Intel MPI etc.

For example, the information about Open MPI can be obtained from http://www.open-mpi.org/

CMake

CMake is used to build G4MPI library, that co-works with Geant4 build system.

Optional

ROOT for histogramming/analysis

How to build G4MPI

To build G4MPI library, use CMake on Geant4 library installed with CMake build.

Follow these commands,

> mkdir build
> cd build
> cmake -DGeant4_DIR=<your Geant4 install path>/lib64/Geant4-V.m.n -DCMAKE_CXX_COMPILER=mpicxx \
 -DCMAKE_INSTALL_PREFIX=<where-G4mpi-lib-will-be-installed> <g4source>/examples/extended/parallel/MPI/source
> make
> make install

Replace mpicxx with your MPI compiler wrapper.

The library and header files will be installed on the installation directory specified in CMAKE_CXX_COMPILER a CMake configuration file will also be installed (see examples on how to compile an application using G4mpi)

How to use

How to make parallel applications

An example of a main program:

#include "G4MPImanager.hh"
#include "G4MPIsession.hh"

int main(int argc,char** argv)
{
  // At first, G4MPImanager/G4MPIsession should be created.
  G4MPImanager* g4MPI= new G4MPImanager(argc,argv);
      
  // MPI session (G4MPIsession) instead of G4UIterminal
  G4MPIsession* session= g4MPI-> GetMPIsession();
      
  // user application setting
  G4RunManager* runManager= new G4RunManager();

  ....

  // After user application setting, just start a MPI session.
  MPIsession treats both interactive and batch modes.
  session-> SessionStart();

  // Finally, terminate the program
  delete g4MPI;
  delete runManager;
}

Notes about session shell

LAM/MPI users can use "G4tcsh" as an interactive session shell. For other users (Open MPI/MPICH2), plesae use G4csh (default).

In case of OpenMPI, LD_LIBRARY_PATH for OpenMPI runtime libraries should be set at run time. Alternatively, you can add this path to the dynamic linker configuration using ldconfig. (needs sys-admin authorization)

MPI runtime Environment

Make hosts/cluster configuration of your MPI environment.
Launch MPI runtime environment, typically executing lamboot (LAM) / mpdboot (MPICH2) / mpd (Intel).

How to run

For example,

> mpiexec -n # <your application>

Instead, mpirun command is more convenient for LAM users.

MPI G4UI commands

G4UI commands handling the G4MPI interface are placed in /mpi/.

Command directory path : /mpi/
    
Guidance :
MPI control commands
    
 Sub-directories :
 Commands :
   verbose *        Set verbose level.
   status *         Show mpi status.
   execute *        Execute a macro file. (=/control/execute)
   beamOn *         Start a parallel run w/ thread.
   .beamOn *        Start a parallel run w/o thread.
   masterWeight *   Set weight for master node.
   showSeeds *      Show seeds of MPI nodes.
   setMasterSeed *  Set a master seed for the seed generator.
   setSeed *        Set a seed for a specified node.

Notes:

While "/run/beamOn" and "/mpi/beamOn" commands invoke beam-on in background, so you can input UI commands even while event processing. Note that drawing tracks in OpenGL with these commands causes a crash. Please use /mpi/.beamOn command instead.

The original "/control/execute" and "/run/beamOn" are overwritten with "/mpi/execute" and "/mpi/beamOn" commands respectively, that are customized for the MPI interface.

Examples

There are a couple of examples for Geant4 MPI applications.

In some cases, you need to set some additional environment variables for running examples:

G4LEDATA : directory path for low energy EM data
G4LEVELGAMMADATA : directory path for photon evapolation
G4SAIDXSDATA : directory path for nucleon cross section data

For using ROOT libraries

ROOTSYS : root path of the ROOT package

exMPI01

A simple application.

Configuration:

Geometry : chamber / calorimeter
Primary : particle gun (200 MeV electron as default)
Physics List : FTFP_BERT

Features:

Particles are transported in a geometry without any scoring.
Learn how to parallelized your G4 session.

exMPI02 (ROOT application)

An example of dosimetry in a water phantom.

Configuration:

Geometry : water phantom
Primary : broad beam (200 MeV proton)
Physics List : FTFP_BERT
Analysis : ROOT histogramming

Features:

Score dose distribution in a water phantom.
Learn how to parallelized your applications.
Create a ROOT file containing histograms/trees in each node.