Import Geant4 4.1.0 source tree
This commit is contained in:
Gabriele Cosmo
@@ -0,0 +1,162 @@
|
||||
$Id: 000README,v 1.3 2002/03/09 09:48:03 cooperma Exp $
|
||||
-------------------------------------------------------------------
|
||||
|
||||
ParGeant4: Geant4/TOP-C, a parallelization of Geant4
|
||||
Version 0.9
|
||||
(event-level parallelism only)
|
||||
|
||||
Gene Cooperman
|
||||
Northeastern University
|
||||
gene@ccs.neu.edu,
|
||||
|
||||
For the latest information on ParGeant4, see:
|
||||
http://www.ccs.neu.edu/home/gene/pargeant4.html
|
||||
Note that a version now exists that runs Geant4 over the Grid.
|
||||
Please write to gene@ccs.neu.edu for further information.
|
||||
To port other applications to a parallel version, read the
|
||||
files ../../info/PAR_INSTALL and ../../info/PAR_README.
|
||||
|
||||
|
||||
This parallelization was done using a precompiled library for Geant4 4.0
|
||||
See the beginning of GNUmakefile for reasonable `make' targets to run it.
|
||||
To run it:
|
||||
0. a. Follow the standard Geant4 installation procedure.
|
||||
b. Verify that the Geant4 sequential examples work for you:
|
||||
cd $G4INSTALL/examples/novice/exampleN04
|
||||
make
|
||||
$G4WORKDIR/bin/$G4SYSTEM/ParN04 ParN04.in
|
||||
c. Download and install TOP-C.
|
||||
The TOP-C home page is at http://www.ccs.neu.edu/home/gene/topc.html
|
||||
cd <TOPC_INSTALL_DIR>
|
||||
gzip -dc topc.tar.gz | tar -xvf -
|
||||
cd topc
|
||||
./configure
|
||||
make
|
||||
make test
|
||||
[ Copy bin/topc-config to your path ]
|
||||
2. make run
|
||||
[ By default, the included `procgroup' file creates two slave processes
|
||||
on localhost. ]
|
||||
[ Note that in addition to output on master,
|
||||
$G4WORKDIR/bin/$G4SYSTEM/slave*.out contains slave output. ]
|
||||
[ To remove intermediate files and start over: make parclean ]
|
||||
3. Try running it with slave processes on remote processes.
|
||||
First, test that your local environment is set up correctly.
|
||||
Try: ssh <REMOTE_HOSTNAME> $G4WORKDIR/bin/$G4SYSTEM/ParN04 `pwd`/ParN04.in
|
||||
The above command needs to work without asking for a password.
|
||||
[ If you use dynamic libraries (*.so), make sure the LD_LIBRARY_PATH
|
||||
in your shell startup file (e.g. ~.tcshrc) includes both:
|
||||
$G4INSTALL/lib/$G4SYSTEM and $CLHEP_BASE_DIR/lib
|
||||
If you use AFS, you may need to type 'klog' to renew your AFS token. ]
|
||||
In `procgroup' file, replace `localhost' by desired remote hosts;
|
||||
Add additional remote hosts (additional slaves) if you like.
|
||||
Then: make run
|
||||
|
||||
============================================================================
|
||||
If you read ParGNUmakefile, you'll find other things that you can
|
||||
modify. For example, all TOP-C additions are in conditionals:
|
||||
remove -DG4USE_TOPC from ParGNUmakefile and:
|
||||
make parclean; make run
|
||||
in order to re-compile and rerun without TOP-C.
|
||||
Define REMOTE_SHELL differently if you don't use `ssh' for a remote shell.
|
||||
(If undefined, ParGNUmakefile defines it to be `ssh')
|
||||
Define MACROFILE diferently to use a different set of input commands.
|
||||
Define MEM_MODEL=--seq
|
||||
to run with TOP-C, but using a single (sequential) process, suitable
|
||||
for easy debugging (via gdb, for example).
|
||||
Try: pushd $G4WORKDIR/bin/$G4SYSTEM/; ./ParN04 --TOPC-help
|
||||
to see TOP-C run-time options that can be invoked, such as
|
||||
pushd $G4WORKDIR/bin/$G4SYSTEM/; ./ParN04 --TOPC-num-slaves=5 ParN04.in
|
||||
Alternatively, modify TOPC_OPTIONS in ParGNUmakefile for the same effect.
|
||||
|
||||
You can also try other targets: make run-debug
|
||||
This will run it under gdb, so you can single step to see what happens.
|
||||
make parclean - Start over with clean set of files.
|
||||
|
||||
============================================================================
|
||||
New or modified files:
|
||||
ParN04.cc - Adds one line: #include "ParN04.icc"
|
||||
ParExample.icc inserts: #include "topc.h"
|
||||
and causes main to calls TOPC_init, TOPC_finalize,
|
||||
and to use: `new ParRunManager' instead of `new G4RunManager'
|
||||
GNUmakefile - Adds one line at beginning: include ParGNUmakefile
|
||||
ParGNUmakefile defines EXTRALIBS and CPPFLAGS so as to
|
||||
modify behavior of config/binmake.gmk
|
||||
in order to use TOP-C libraries and includes
|
||||
procgroup - Specifies which slave hosts to use, and where to put output
|
||||
For example: localhost 1 - > slave1.out
|
||||
host=`localhost', executable=`same as master',
|
||||
params of slave=`> slave1.out' (redirect output)
|
||||
If output not redirected, it goes to stdout on master.
|
||||
src/ParRunManager.cc - ParRunManger derived from G4RunManager
|
||||
replaces Gr4RunManager::DoEventLoop w/ TOP-C parallel loop,
|
||||
Adds certain local vars of DoEventLoop as ParRunManager members
|
||||
src/ParMarshaledObj.cc - run-time utilities for marshalling
|
||||
src/ParExN04MarshaledHits.cc - marshals N04 hits (calorimeter & tracking hits)
|
||||
include/Par*.hh - Currently just ParRunManager.hh
|
||||
~/slave*.out - Contains outputs of slave1, slave2, etc.
|
||||
Generated each time parallel ParN04 is executed.
|
||||
These files are specified in the file procgroup.
|
||||
thwap - A script I inherited from our N.U. systems group
|
||||
`thwap ParN04' will clobber all processes you own
|
||||
whose command line includes `ParN04' as substring.
|
||||
Handy for runaway processes when doing parallelism.
|
||||
|
||||
====================================================================
|
||||
This version passes an event number to the slave and lets the
|
||||
slave generate the event. The slave passes back marshaled hits to
|
||||
the master.
|
||||
|
||||
I will integrate the track level parallelism into this scenario at
|
||||
a later date. For the track level, I will generate several
|
||||
secondary tracks on the master, and then convert the secondary tracks
|
||||
to new events that can be passed to slaves. I will do this only if
|
||||
I detect that there are not enough initial events to fully occupy all
|
||||
the slaves. This scheme has the drawback that we are splitting an event
|
||||
into many events, which may make the summarization, histogram, and so
|
||||
on more difficult. However, track level parallelism will be triggered
|
||||
only when a very small number of events are generated.
|
||||
|
||||
I also want to support postponing
|
||||
a track to the next event ( G4ClassificationOfNewTrack::fPostpone .
|
||||
To do this, each slave will wait to retire an event until it knows that
|
||||
the previous event has been retired.
|
||||
|
||||
In addition, I plan to have only the master read commands and pass
|
||||
them to the slaves. Currently, the master and slaves each read
|
||||
identical commands.
|
||||
|
||||
====================================================================
|
||||
If you are curious about some of the layers, the following
|
||||
stack trace [somewhat out of date now] gives some idea.
|
||||
G4RunManager::BeamOn calls ParRunManager::DoEventLoop
|
||||
(since G4RunManager::DoEventLoop is virtual)
|
||||
ParRunManager::DoEventLoop calls TOPC_master_slave
|
||||
TOPC_master_slave calls submit_task_input
|
||||
submit_task_input eventually calls COMM_send_msg which calls MPI_Send
|
||||
(COMM_send_msg is the communication layer of TOPC;
|
||||
The exampleN02.cc was linked with the TOP-C MPI communication layer.
|
||||
The same source could have been linked with a POSIX threads layer,
|
||||
a communication layer, or some other communication layer.
|
||||
)
|
||||
MPI_send calls send
|
||||
(where send is the socket system call of libc.so)
|
||||
|
||||
(gdb) where
|
||||
#0 0x41946c62 in send () from /lib/libc.so.6
|
||||
#1 0x400839c1 in send () at wrapsyscall.c:186
|
||||
#2 0x805c547 in MPI_Send (buf=0x82690fc, count=4, datatype=3, dest=2, tag=1, comm=0) at sendrecv.c:236
|
||||
#3 0x805a0b5 in COMM_send_msg (msg=0x82690fc, msg_size=4, dst=2, tag=TASK_INPUT_TAG) at comm-mpi.c:224
|
||||
#4 0x805774e in send_task_input (slave=2, input={data = 0x82690fc, data_size = 4}, tag=TASK_INPUT_TAG) at topc.c:560
|
||||
#5 0x8057aa8 in submit_task_input (input={data = 0x82690fc, data_size = 4}) at topc.c:659
|
||||
#6 0x805813c in TOPC_master_slave (generate_task_input_=0x4003d2e4 <ParRunManager::GenerateEventInput(void)>,
|
||||
do_task_=0x4003d350 <ParRunManager::DoEvent(int *)>, check_task_result_=0x4003d420 <ParRunManager::CheckEventResult(int *, void *)>,
|
||||
update_shared_data_=0) at topc.c:922
|
||||
#7 0x4003d18c in ParRunManager::DoEventLoop (this=0x80c0bf0, n_event=1, macroFile=0x0, n_select=-1) at src/ParRunManager.cc:51
|
||||
#8 0x400b14d1 in G4RunManager::BeamOn () from /afs/cern.ch/user/c/cooperma/scratch-pcitapi07/geant4/lib/libG4run.so
|
||||
#9 0x400b870a in G4RunMessenger::SetNewValue () from /afs/cern.ch/user/c/cooperma/scratch-pcitapi07/geant4/lib/libG4run.so
|
||||
#10 0x4167157b in G4UIcommand::DoIt () from /afs/cern.ch/user/c/cooperma/scratch-pcitapi07/geant4/lib/libG4intercoms.so
|
||||
#11 0x416810a3 in G4UImanager::ApplyCommand () from /afs/cern.ch/user/c/cooperma/scratch-pcitapi07/geant4/lib/libG4intercoms.so
|
||||
#12 0x805db7e in G4UIterminal::ExecuteCommand () at /afs/cern.ch/sw/lhcxx/specific/redhat61/3.2.0/include/CLHEP/Random/Randomize.h:64
|
||||
#13 0x805d42d in G4UIterminal::SessionStart () at /afs/cern.ch/sw/lhcxx/specific/redhat61/3.2.0/include/CLHEP/Random/Randomize.h:64
|
||||
#14 0x8056a3d in main (argc=1, argv=0x80bfa00) at ParN04.cc:98
|
||||
@@ -0,0 +1,28 @@
|
||||
$Id: BUG.userStackingAction,v 1.1 2002/03/05 15:22:07 gcosmo Exp $
|
||||
-------------------------------------------------------------------
|
||||
When Geant4 4.0 sees empty event, it still calls userStackingAction.
|
||||
|
||||
In the function:
|
||||
G4StackManager::PopNextTrack(G4VTrajectory**newTrajectory)
|
||||
|
||||
in Geant4 4.0, one has:
|
||||
|
||||
145 if(userStackingAction) userStackingAction->NewStage();
|
||||
146 #ifdef G4VERBOSE
|
||||
147 if( verboseLevel > 0 ) G4cout << " " << GetNUrgentTrack()
|
||||
148 << " urgent tracks and " << GetNWaitingTrack()
|
||||
149 << " waiting tracks." << G4endl;
|
||||
150 #endif
|
||||
151 if( ( GetNUrgentTrack()==0 ) && ( GetNWaitingTrack()==0 ) ) return 0;
|
||||
|
||||
It seems to me that the call, `return 0' (line 151), should occur before
|
||||
the call to `NewStage()' (line 145). If there are no tracks to
|
||||
be stacked (neither GetNUrgentTrack() nor GetNWaitingTrack()),
|
||||
then there is no new stage. So, it's a mistake to call
|
||||
userStackingAction->NewStage().
|
||||
|
||||
This affects my parallelization of Geant4, in which I need to purposely
|
||||
create a trivial event on the master, and call
|
||||
G4EventManager()::ProcessOneEvent() on it to initialize the event on the
|
||||
master. I am doing this in order to copy the hits from the slave into
|
||||
an event on the master.
|
||||
@@ -0,0 +1,48 @@
|
||||
$Id: README,v 1.1 2002/03/05 15:22:08 gcosmo Exp $
|
||||
-------------------------------------------------------------------
|
||||
|
||||
=========================================================
|
||||
Geant4 - an Object-Oriented Toolkit for Simulation in HEP
|
||||
=========================================================
|
||||
|
||||
ParN04
|
||||
------
|
||||
|
||||
ParN04 has a simplified collider detector geometry.
|
||||
This example demonstrates the following features.
|
||||
[ Read 000README for a description of how to run it in parallel. ]
|
||||
|
||||
1. PYTHIA primary events.
|
||||
|
||||
ExN04PrimaryGeneratorAction has G4HEPEvtInterface as the generator.
|
||||
G4HEPEvtInterface accesses to "pythia_event.data", which contains three
|
||||
events of Higgs generation produced by PYTHIA. "pythia_main.f" is an
|
||||
example FORTRAN code of PYTHIA for generating this event sample.
|
||||
|
||||
2. Readout geometry
|
||||
|
||||
ExN04DetectorConstruction defines a simplified collider detecor
|
||||
geometry, tracker made of cylindrical tubes, calorimeter made of
|
||||
cylindrical tubes, and muon trackers made of planes.
|
||||
Cylindrical calorimeter is made of tubes of lead and scintirator
|
||||
without cut in phi nor z direction. Energy deposition in scintirator
|
||||
is accumurated by ExN04CalorimeterSD sensitive detector, which has
|
||||
a readout geometry to find the phi-z cell.
|
||||
|
||||
3. Full set of "ordinary" physics processes
|
||||
|
||||
ExN04PhysicsList defines almost all of leptons and hadrons which
|
||||
Geant4 has dedicated classes for. Also almost all physics processes
|
||||
Geant4 has are defined.
|
||||
|
||||
4. Event filtering by the stacking mechanism.
|
||||
|
||||
Higgs events in "pythia_event.data" have two lepton pairs produced
|
||||
by the Higgs decay via Z0. At the first stage of each event, only the
|
||||
primary muons are tracked without tracking secondaries. then the number
|
||||
of hits on the muon trackers are examined. At the next stage, only
|
||||
the primary charged particles are tracked only inside the barrel
|
||||
tracking area and the isolation of the primary muons are examined.
|
||||
At the third stage, all particles in the RoI (Region of Interest) along
|
||||
the isolated muons are tracked. All these examinations are applied in
|
||||
ExN04StackingAction.
|
||||
Reference in New Issue
Block a user