Gabriele Cosmo
331 lines
14 KiB
Markdown
331 lines
14 KiB
Markdown
\page ExamplePar04 Example Par04
|
|
|
|
This example demonstrates how to use the Machine Learning (ML) inference
|
|
to create energy deposits as a fast simulation model using
|
|
<a href="https://github.com/microsoft/onnxruntime">ONNX runtime</a>,
|
|
<a href="https://github.com/lwtnn/lwtnn">LWTNN</a>, and
|
|
<a href="https://pytorch.org/cppdocs/frontend.html">LibTorch</a> libraries.
|
|
|
|
The model used in this example was trained externally (in Python) on data
|
|
from this examples' full simulation and can be applied to perform fast simulation.
|
|
The python scripts are available in the training folder.
|
|
|
|
The geometry used in the example is a cylindrical setup of layers: tungsten
|
|
absorber and silicon as the active material. 3D readout geometry (cylindrical)
|
|
is defined dynamically, based on the particle direction at the entrance to the
|
|
calorimeter. This is set using a fast simulation model that is triggered at
|
|
detector entrance. Analysis of energy deposits is done in the event action,
|
|
ntuple with hits is stored.
|
|
|
|
## 1. Detector description
|
|
|
|
The detector consists of cylindrical layers of passive and active material,
|
|
tungsten and silicon, respectively.
|
|
|
|
Fast simulation is attached to the region of the detector.
|
|
|
|
Input macro can specify which layer is considered an active layer (sensitive
|
|
detector is attached to it). For fast simulation both layers should be marked
|
|
as sensitive. It is connected to the way the deposits are created: position is
|
|
centre of the layer, which may often fall within the absorber (which is thicker
|
|
than the active material). In a realistic detector setup, the positions used in
|
|
fast simulation would be calculated properly, to deposit energy within the active
|
|
material.
|
|
|
|
## 2. Sensitive detector
|
|
|
|
- 2.1. Par04SensitiveDetector
|
|
|
|
This SD scores energy originating from showers, in a cylinder around the particle
|
|
direction and position in the calorimeter.
|
|
Sensitive detector inherits from both base classes:
|
|
- G4VSensitiveDetector: for processing of detailed/non-fast simulation hits
|
|
- G4VFastSimSensitiveDetector: for processing of fast sim (G4FastSim) hits.
|
|
Hits are placed in the same hit collection, with a different flag to distinguish
|
|
between those originated in the full simulation, and those from the fast
|
|
simulation.
|
|
During visualisation, hits are represented as volumes of different colour:
|
|
green for full simulation and red for fast simulation.
|
|
|
|
- 2.2. Par04ParallelFullSensitiveDetector
|
|
|
|
This SD represents a physical readout structure to the detector (a regular grid).
|
|
UI settings are available to set number of slices (azimuthal segmentation) and number
|
|
of rows (segmentation along beam axis). Number of layers cannot be changed as it
|
|
corresponds to the number of layers placed at the detector construction time. Only
|
|
deposits in the active (sensitive) layers are scored in this SD.
|
|
|
|
- 2.2. Par04ParallelFastSensitiveDetector
|
|
|
|
This SD represents a physical readout that takes into account deposits originating
|
|
from fast simulation, so cells span over active and passive layers. This allows to
|
|
account all energy from the parameterisation.
|
|
|
|
## 3. Primary generation
|
|
|
|
Particle gun is used as a primary generator. 10 GeV electron is used by default.
|
|
By default particles are generated along y axis. Those values
|
|
can be changed using /gun/ UI commands.
|
|
|
|
## 4. Physics List
|
|
|
|
FTFP_BERT modular physics list is used. On top of it, fast simulation physics
|
|
is registered for selected particles (electrons, positrons).
|
|
|
|
|
|
## 5. User actions
|
|
|
|
- Par04RunAction : run action used for initialization and termination
|
|
of the run. Histograms for analysis of shower development
|
|
in the detector are created.
|
|
|
|
- Par04EventAction : event action used for initialization and termination
|
|
of the event. Analysis of shower development is performed
|
|
on event-by-event basis.
|
|
|
|
## 6. ML Inference
|
|
|
|
- Par04MLFastSimModel : model used for parametrisation of electrons, positrons,
|
|
and gammas. Energy is deposited and
|
|
distributed according to inferred values from the ML model.
|
|
This class triggers the inference setup, asks for values,
|
|
and deposits energies at given positions.
|
|
|
|
- Par04InferenceSetup : this class is used to initialize the inference parameters
|
|
(user application specific) such as the inference library,
|
|
the path and name of the inference model and the size of
|
|
the input inference vector(latent dimension and and condition size).
|
|
This class constructs this vector and triggers the interface
|
|
corresponding to the specified input inference library.
|
|
After the inference, the post processing step consists of
|
|
scaling back inferred values to the original range.
|
|
|
|
- Par04InferenceInterface : is a base class that allows to read in the ML model, configure
|
|
and execute inference.
|
|
|
|
- Par04OnnxInference and Par04LWTNNInference and Par04TorchInference : inference library specific
|
|
classes that inherit from the base class Par04InferenceInterface.
|
|
|
|
|
|
## 7. Output
|
|
|
|
The execution of the program (examplePar04) produces an output with histograms.
|
|
Ntuples are also stored. They are not merged if the application is run on multiple threads.
|
|
|
|
The macro file examplePar04.mac is used to run full simulation. It will simulate 100
|
|
events, for single 10 GeV electron beams.
|
|
If CMake is able to find inference libraries (LWTNN and/or ONNX Runtime and/or LibTorch), a configuration
|
|
macro will be available for that library (examplePar04_lwtnn_vae.mac and/or examplePar04_onnx_vae.mac
|
|
and/or examplePar04_torch_vae.mac and/or examplePar04_onnx_calodit.mac and/or
|
|
examplePar04_torch_calodit.mac). It will use a trained model to run inference and create showers
|
|
in the detector by directly depositing energy.
|
|
|
|
There are two models available VAE and CaloDiT-2. CaloDiT-2 is a more sophisticated transformer-based
|
|
diffusion model which gives much better accuracy, especially on the cell energy distribution, and
|
|
also can be easily adapted to new detectors.
|
|
Notes for CaloDiT-2; first, it operates on a lower granular cylindrical virtual mesh than VAE (which became
|
|
from this release also the default for full simulation).
|
|
Second, we do not support LWTNN inference, as PyTorch to LWTNN conversion is not straightforward.
|
|
|
|
## 8. How to build and run the example
|
|
|
|
- LWTNN, ONNX Runtime, and LibTorch are available on LCG. In order to use them, you can set a `CMAKE_PREFIX_PATH`:
|
|
```
|
|
% source /cvmfs/sft.cern.ch/lcg/contrib/gcc/11.3.0/x86_64-centos7/setup.sh
|
|
% cmake -DCMAKE_PREFIX_PATH="/cvmfs/sft.cern.ch/lcg/releases/LCG_102b/lwtnn/2.11.1/x86_64-centos7-gcc11-opt/;/cvmfs/sft.cern.ch/lcg/releases/LCG_102b/onnxruntime/1.11.1/x86_64-centos7-gcc11-opt/;/cvmfs/sft.cern.ch/lcg/releases/LCG_102b/torch/1.11.0/x86_64-centos7-gcc11-opt/lib/python3.9/site-packages/torch/" <Par04_SOURCE>
|
|
```
|
|
|
|
- Compile and link to generate the executable (in your CMake build directory):
|
|
```
|
|
% cmake <Par04_SOURCE>
|
|
% make
|
|
```
|
|
|
|
- Execute the application (in batch mode):
|
|
```
|
|
% ./examplePar04 -m examplePar04.mac
|
|
```
|
|
which produces two root file for full simulation.
|
|
|
|
- Execute the application (in interactive mode):
|
|
```
|
|
% ./examplePar04 -i -m vis.mac
|
|
```
|
|
which allows to visualize hits (from full simulation).
|
|
|
|
- If ONNX Runtime is available:
|
|
```
|
|
% ./examplePar04 -m examplePar04_onnx_vae.mac
|
|
% ./examplePar04 -m examplePar04_onnx_calodit.mac
|
|
```
|
|
For interactive mode with visualization:
|
|
```
|
|
% ./examplePar04 -i -m vis_onnx_vae.mac
|
|
% ./examplePar04 -i -m vis_onnx_calodit.mac
|
|
```
|
|
|
|
- If LWTNN is available:
|
|
```
|
|
% ./examplePar04 -m examplePar04_lwtnn_vae.mac
|
|
```
|
|
For interactive mode with visualization:
|
|
```
|
|
% ./examplePar04 -i -m vis_lwtnn_vae.mac
|
|
```
|
|
|
|
- If LibTorch is available:
|
|
```
|
|
% ./examplePar04 -m examplePar04_torch_vae.mac
|
|
% ./examplePar04 -m examplePar04_torch_calodit.mac
|
|
```
|
|
For interactive mode with visualization:
|
|
```
|
|
% ./examplePar04 -i -m vis_torch_vae.mac
|
|
% ./examplePar04 -i -m vis_torch_calodit.mac
|
|
```
|
|
|
|
- Additional options available:
|
|
```
|
|
% ./examplePar04 -m examplePar04.mac -r 0
|
|
```
|
|
For serial run manager mode
|
|
```
|
|
% ./examplePar04 -m examplePar04.mac -r 1 -t 8
|
|
```
|
|
For multi-threaded run manager mode with 8 threads
|
|
```
|
|
% ./examplePar04 -m examplePar04.mac -r 2
|
|
```
|
|
For tasking run manager mode with number of tasks that can be change via env variable G4FORCE_EVENTS_PER_TASK
|
|
|
|
By default, CMake will attempt to build fast simulation with ONNX Runtime and LWTNN. However, if none
|
|
of those libraries is found, it will proceed with full simulation only. The search can be switched
|
|
off manually switching CMake flag `INFERENCE_LIB` to `OFF` (`-DINFERENCE_LIB=OFF`)
|
|
|
|
## 9. Macros
|
|
|
|
common_settings_lowgran.mac - A macro with common settings, executed by all other macros that use low granularity
|
|
(e.g. detector settings). This can be used directly by fast simulation, and for full sim
|
|
the sensitivity of absorber must be set to false (it's done in examplePar04.mac or vis.mac).
|
|
|
|
common_settings_highgran.mac - A macro with common settings, executed by all other macros that use high granularity
|
|
(e.g. detector settings). This can be used directly by fast simulation, and for full sim
|
|
the sensitivity of absorber must be set to false.
|
|
|
|
common_settings_vis.mac - A macro with common settings, executed by all visualisation macros.
|
|
|
|
common_settings_postInit.mac - A macro with common settings, executed after initialization, e.g. for particle gun settings.
|
|
|
|
vis.mac - Allows to run visualization. Pass it to the example in interactive mode ("-i" passed to the executable).
|
|
It can be used to visualize full simulation. Lower granularity is used for visualisation. To be compared to CaloDiT-2.
|
|
|
|
vis_onnx_vae.mac - Allows to run visualization with ONNX Runtime inference using VAE. Pass it to the example in interactive mode
|
|
("-i" passed to the executable). It contains necessary settings of the inference.
|
|
|
|
vis_lwtnn_vae.mac - Allows to run visualization with LWTNN inference using VAE. Pass it to the example in interactive mode
|
|
("-i" passed to the executable). It contains necessary settings of the inference.
|
|
|
|
vis_torch_vae.mac - Allows to run visualization with LibTorch inference using VAE. Pass it to the example in interactive mode
|
|
("-i" passed to the executable). It contains necessary settings of the inference.
|
|
|
|
examplePar04.mac - Runs full simulation. It will run 100 events with single electrons, 10 GeV and
|
|
along y axis. Lower granularity is used, to be compared with CaloDiT-2.
|
|
|
|
examplePar04_onnx_vae.mac - Available only if ONNX Runtime is found by CMake. Runs fast simulation with
|
|
a NN stored in onnx file for VAE.
|
|
|
|
examplePar04_lwtnn_vae.mac - Available only if LWTNN is found by CMake. Runs fast simulation with
|
|
a NN stored in json file for VAE.
|
|
|
|
examplePar04_torch_vae.mac - Available only if LibTorch is found by CMake. Runs fast simulation with
|
|
a NN stored in pt file for VAE.
|
|
|
|
vis_onnx_calodit.mac - Allows to run visualization with ONNX Runtime inference using CaloDiT-2.
|
|
|
|
vis_torch_calodit.mac - Allows to run visualization with LibTorch inference using CaloDiT-2.
|
|
|
|
examplePar04_onnx_calodit.mac - Available only if ONNX Runtime is found by CMake. Runs fast simulation with
|
|
a NN stored in onnx file for CaloDiT-2.
|
|
|
|
examplePar04_torch_calodit.mac - Available only if LibTorch is found by CMake. Runs fast simulation with
|
|
a NN stored in pt file for CaloDiT-2.
|
|
|
|
## 10. UI commands
|
|
|
|
UI commands useful in this example:
|
|
|
|
- activation/disactivation of the fast simulation model:
|
|
```
|
|
/param/ActivateModel inferenceModel
|
|
/param/InActivateModel inferenceModel
|
|
```
|
|
|
|
- particle gun commands
|
|
```
|
|
/gun/particle e-
|
|
/gun/energy 10 GeV
|
|
/gun/direction 0 1 0
|
|
/gun/position 0 0 0
|
|
```
|
|
|
|
UI commands defined in this example:
|
|
- detector settings
|
|
```
|
|
/Par04/detector/setDetectorInnerRadius 80 cm
|
|
/Par04/detector/setDetectorLength 2 m
|
|
/Par04/detector/setNbOfLayers 90
|
|
/Par04/detector/setAbsorber 0 G4_W 1.4 mm false
|
|
/Par04/detector/setAbsorber 1 G4_Si 0.3 mm true
|
|
```
|
|
|
|
- readout mesh
|
|
```
|
|
/Par04/mesh/setSizeOfRhoCells 2.325 mm # (4.65 for CaloDiT-2)
|
|
/Par04/mesh/setSizeOfZCells 3.4 mm
|
|
/Par04/mesh/setNbOfRhoCells 18 # (9 for CaloDiT-2)
|
|
/Par04/mesh/setNbOfPhiCells 50 # (16 for CaloDiT-2)
|
|
/Par04/mesh/setNbOfZCells 45
|
|
```
|
|
|
|
- inference setup
|
|
```
|
|
/Par04/inference/setSizeLatentVector 10
|
|
/Par04/inference/setSizeConditionVector 4
|
|
/Par04/inference/setModelPathName MLModels/Generator.onnx # (or cd.onnx for CaloDiT-2)
|
|
/Par04/inference/setProfileFlag 0
|
|
/Par04/inference/setOptimizationFlag 0
|
|
/Par04/inference/setInferenceLibrary ONNX
|
|
/Par04/inference/setSizeOfRhoCells 2.325 mm # (4.65 for CaloDiT-2)
|
|
/Par04/inference/setSizeOfZCells 3.4 mm
|
|
/Par04/inference/setNbOfRhoCells 18 # (9 for CaloDiT-2)
|
|
/Par04/inference/setNbOfPhiCells 50 # (16 for CaloDiT-2)
|
|
/Par04/inference/setNbOfZCells 45
|
|
```
|
|
|
|
## 11. Python scripts for training
|
|
|
|
The scripts available in the training folder were used to firstly convert
|
|
the ROOT files to the h5 files, preprocess the data and then train
|
|
the VAE model of this example. More details can be found in README in
|
|
training_vae (\ref refPar04training_vae).
|
|
|
|
For CaloDiT-2 training and adaptation to new detectors, refer README.md in
|
|
training_calodit (\ref refPar04training_calodit).
|
|
|
|
## 12. Public data
|
|
|
|
Data generated with full simulation with this example has been published on
|
|
<a href="https://doi.org/10.5281/zenodo.6082201">zenodo</a>.
|
|
It was used (as well as VAE) for this publication:
|
|
<a href="https://doi.org/10.1016/j.physletb.2023.138079">doi.org/10.1016/j.physletb.2023.138079</a>.
|
|
|
|
Data generated with low granularity (so-called dataset2) and high granularity (so-called dataset3) are
|
|
released for the CaloChallenge:
|
|
dataset2 (lowgran): <a href="https://doi.org/10.5281/zenodo.6366271">doi.org/10.5281/zenodo.6366271</a>.
|
|
dataset3 (highgran): <a href="https://doi.org/10.5281/zenodo.6366324">doi.org/10.5281/zenodo.6366324</a>.
|
|
|
|
## See more
|
|
|
|
- \subpage refPar04training_calodit
|
|
- \subpage refPar04training_vae
|