lars
2358a75ee1
CI / Sync project version with tag (pull_request) Has been skipped
CI / Format (ruff format) (pull_request) Successful in 43s
CI / Lint (ruff check) (pull_request) Successful in 54s
CI / Type check (ty) (pull_request) Successful in 53s
CI / Tests (pull_request) Successful in 2m33s
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
CI / Publish package to Gitea package registry (pull_request) Has been skipped
Closes the roadmap's long-standing "no eval-latency number exists for any configuration" gap. Instruments `giant rollout` to record per-physical-step wall-clock cost in its YAML sidecar, adds a measured Geant4/miniCaloSim per-step reference (giant/analysis/geant4_reference.py, from a 3-energy, 4-event-count-per-energy local benchmark), and wires both into a new eval_cost_per_step PlotSpec in the giant analyze gallery. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
77 lines
3.6 KiB
Python
77 lines
3.6 KiB
Python
"""Measured Geant4 (miniCaloSim) per-step eval cost — the reference line for
|
|
``eval_cost_per_step`` in ``catalog.py``.
|
|
|
|
Mirrors the precedent set by ``runtime_estimate.py``'s ``_COST_MODEL``: a
|
|
constant table measured once on a specific machine and pasted in, with the
|
|
methodology and provenance recorded in this docstring rather than derived at
|
|
runtime (there is no live Geant4 install on the machines that run
|
|
``giant analyze``, and re-measuring per invocation would be both slow and
|
|
noisy — see the module docstring precedent).
|
|
|
|
**Methodology** (``scratchpad/bench_geant4.py``, a one-off, not a `dwarf`
|
|
subcommand): ``run_pbwo4`` (the default homogeneous-PbWO4 miniCaloSim
|
|
executable, see ``~/Programming/minicalosim``) was timed at 3 beam energies
|
|
(1/10/50 GeV) and **4 event counts each**, converting each run's ROOT output
|
|
to Parquet with ``giant.tools.steps_to_parquet.convert_steps_to_parquet``
|
|
immediately after. Event counts were scaled down as energy rose (100/400/
|
|
1000/2000 at 1 GeV, 30/100/200/300 at 10 GeV, 10/25/45/60 at 50 GeV) to keep
|
|
every run's row count under ~8.1M — a naive 50/200 pair at 50 GeV produces
|
|
~27M steps and OOM'd the conversion step on a 14GB laptop. Per-energy linear
|
|
fits (``t = intercept + slope * n``) separate Geant4's one-time init (physics
|
|
tables, geometry construction) from its true marginal per-event cost — the
|
|
slope, not a naive ``t / n_events`` from a single run, is what feeds
|
|
``sim_us_per_step`` below. The per-step denominator is the produced
|
|
``Steps``-tree/Parquet row count, matching the "physical step" unit
|
|
``giant rollout``'s ``timing.n_physical_rows`` uses on the surrogate side.
|
|
Both stages ran single-threaded (default Geant4 threading), pinned to one
|
|
CPU core.
|
|
|
|
``sim_us_per_step``/``convert_us_per_step``/``sim_ms_per_event`` below are
|
|
the mean across the 3 energies. With 4 event-count points per energy (up
|
|
from an initial 2-point pass, which had ~80% spread and nonsensical negative
|
|
fitted intercepts at 10/50 GeV — an artifact of extrapolating a 2-point
|
|
line), both quantities are now energy-flat as physically expected:
|
|
``sim_us_per_step`` spread ~5%, ``convert_us_per_step`` spread ~13.5%. Treat
|
|
these as reliable to about that precision.
|
|
|
|
**Caveat — hardware asymmetry**: this reference is single-core CPU. A
|
|
surrogate rollout's ``timing`` block will typically be measured on a batched
|
|
GPU. The resulting ratio in ``eval_cost_per_step`` is a *deployment* speedup
|
|
(what you'd actually see swapping Geant4 for the surrogate in a production
|
|
pipeline), not a same-hardware or per-FLOP comparison — state this whenever
|
|
quoting the number.
|
|
|
|
**Staleness**: re-run ``scratchpad/bench_geant4.py`` (and update this file)
|
|
if measured on different hardware, after a miniCaloSim/Geant4 version bump,
|
|
or if this reference is more than a year or two stale.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
GEANT4_REFERENCE: dict = {
|
|
"sim_us_per_step": 11.2903,
|
|
"convert_us_per_step": 11.1014,
|
|
"sim_ms_per_event": 609.6848,
|
|
"provenance": {
|
|
"cpu": "AMD Ryzen 7 PRO 4750U with Radeon Graphics",
|
|
"geant4_version": "11.4.1",
|
|
"minicalosim_sha": "ea917da",
|
|
"measured": "2026-08-31",
|
|
"energies_gev": [1.0, 10.0, 50.0],
|
|
"spread_pct_sim": 4.96,
|
|
"spread_pct_convert": 13.52,
|
|
"threads": 1,
|
|
},
|
|
}
|
|
|
|
|
|
def geant4_per_step_us() -> dict[str, float]:
|
|
"""Sim / convert / total microseconds per physical step, from ``GEANT4_REFERENCE``."""
|
|
sim = GEANT4_REFERENCE["sim_us_per_step"]
|
|
convert = GEANT4_REFERENCE["convert_us_per_step"]
|
|
return {
|
|
"sim_us_per_step": sim,
|
|
"convert_us_per_step": convert,
|
|
"total_us_per_step": sim + convert,
|
|
}
|