Rewrite analysis as streaming rollout-vs-reference plotting pipeline
Replace the monolithic giant/analysis.py (predict-local + RolloutVsTruth
diagnostics) with a lean giant/analysis/ package that compares one
autoregressive `giant rollout` for a checkpoint against a held-out
miniCaloSim reference file, and generates publication-styled plots in
parallel on HTCondor.
Rollout output and a raw reference file share a world-frame physical
column subset under identical names, so the old ALR/local-frame decode
machinery is gone — everything is world-frame mm/MeV.
- sources.py: canonical LazyFrames, synthetic-termination-row filtering,
the secondary view (rollout generation>0 tracks vs reference sec_*_list).
- reduce.py: streaming primitives — a single hist1d group_by pass, per-event
scalars, edep-weighted depth/transverse profiles, species share, leakage.
- context.py/grouping.py: prep resolves fixed bin edges + energy/pdg/material
group sets once into shared.json, so each compute job is one pass, no range
scan (histogram efficiency).
- catalog.py: declarative PlotSpec registry — marginals x {overall,energy,pdg,
material}, per-event totals, shower profiles, species/leakage, secondaries.
- render.py: the only plotstyle/LaTeX importer; PDFs + gallery metadata.
- condor.py + `giant analyze` CLI (prep/compute-one/list/render/submit):
one job per plot, compute/render split (workers polars-only, no LaTeX).
Styling via ETPlot's plotstyle (added to the analysis extra). New tests cover
the reduce primitives, catalog id uniqueness + compute, condor submit, and a
guarded render smoke test. Delete the two predict-diagnostics notebooks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -39,6 +39,9 @@ analysis = [
|
||||
"matplotlib>=3.8,<4",
|
||||
"polars>=1.0,<2",
|
||||
"ipykernel>=7.3.0",
|
||||
# ETPlot's plotstyle (KIT matplotlib theme) + gallery CLI. Only the local
|
||||
# `giant analyze render` step imports it; compute workers never do.
|
||||
"gallery[plotting]",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
@@ -65,6 +68,10 @@ torch = [
|
||||
{ index = "pytorch-cpu", extra = "cpu" },
|
||||
{ index = "pytorch-cu118", extra = "cuda" },
|
||||
]
|
||||
# ETPlot ships the `gallery` distribution (which provides the `plotstyle`
|
||||
# package under its `plotting` extra). Local checkout next to this repo; swap
|
||||
# for `{ git = "https://git.larsbogner.de/lars/ETPlot" }` off-machine.
|
||||
gallery = { path = "../ETPlot", editable = true }
|
||||
|
||||
[[tool.uv.index]]
|
||||
name = "pytorch-cpu"
|
||||
|
||||
Reference in New Issue
Block a user