28 Commits

Author SHA1 Message Date
gitea-actions bb8d16caba chore: update changelog for v0.3.11 [skip ci] 2026-08-26 12:33:51 +00:00
gitea-actions c8a1b4f25d chore: bump version 0.3.10 -> 0.3.11 [skip ci] 2026-08-26 12:33:46 +00:00
lars 23efd6d9ff Merge pull request 'feat(analysis): per-step secondary multiplicity plots' (#85) from analysis/per-step-secondary-multiplicity into master
CI / Sync project version with tag (push) Has been skipped
CI / Lint (ruff check) (push) Successful in 28s
CI / Format (ruff format) (push) Successful in 38s
CI / Type check (ty) (push) Successful in 40s
CI / Tests (push) Successful in 3m37s
CI / Bump version, tag, and update changelog on merge to master (push) Successful in 2m4s
Reviewed-on: #85
2026-08-26 14:28:16 +02:00
lars 9fa6420183 feat(analysis): per-step secondary multiplicity plots
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 2m3s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 1m19s
CI / Lint (ruff check) (push) Successful in 3m55s
CI / Format (ruff format) (push) Successful in 3m54s
CI / Format (ruff format) (pull_request) Successful in 5m37s
CI / Lint (ruff check) (pull_request) Successful in 5m42s
CI / Tests (push) Successful in 8m42s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Tests (pull_request) Successful in 5m3s
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
Replace the event-level n_sec confusion matrix with two step-resolved
secondary-multiplicity comparisons:

- sec_count_per_step: overlay histogram of how many secondaries a single
  step emits, rollout series vs reference.
- sec_count_per_step_by_species: heatmap of per-step multiplicity of one
  species (zero row included) against species, drawn as one panel per
  rollout plus a reference panel, raw counts on a log color scale.

Both are backed by a new sources.secondaries_by_step view, which tags each
secondary with its emitting step — (event_id, parent_id, birth position)
on the rollout side, the row index on the reference side — so neither plot
needs a join against the step frame. Steps that emitted nothing are
recovered by subtraction from the chunk's step count, keeping both specs
sum-mergeable across condor chunks.

The rollout multiplicity is derived from the actual secondary birth rows
rather than the n_sec_pred column, which records the predicted count
before the per-event max-tracks cap.

_render_heatmap gained reference-panel and log-color support;
marginal_distance_summary sets neither key and is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 14:13:18 +02:00
gitea-actions b66574877b chore: update changelog for v0.3.10 [skip ci] 2026-08-26 08:15:52 +00:00
gitea-actions 9b77e04731 chore: bump version 0.3.9 -> 0.3.10 [skip ci] 2026-08-26 08:15:51 +00:00
lars 8dee2feab7 Merge pull request 'docs: bring README and CLAUDE.md in line with v0.3.9' (#82) from docs/sync-readme-claude-md into master
CI / Lint (ruff check) (push) Successful in 38s
CI / Format (ruff format) (push) Successful in 37s
CI / Type check (ty) (push) Successful in 38s
CI / Sync project version with tag (push) Has been skipped
CI / Tests (push) Successful in 2m49s
CI / Bump version, tag, and update changelog on merge to master (push) Successful in 33s
Reviewed-on: #82
2026-08-26 10:05:54 +02:00
lars f2da0642b2 docs: bring README and CLAUDE.md in line with v0.3.9
CI / Format (ruff format) (push) Successful in 36s
CI / Lint (ruff check) (push) Successful in 41s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 32s
CI / Lint (ruff check) (pull_request) Successful in 49s
CI / Format (ruff format) (pull_request) Successful in 48s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 40s
CI / Tests (push) Successful in 5m50s
CI / Tests (pull_request) Successful in 4m36s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
CLAUDE.md still described the pre-v0.3.0 codebase: the Stage-2
autoregressive redesign as "designed, not implemented", a monolithic
network.py, a single global model.conditioning switch, and WGAN as
"implemented, not yet tested".

- Architecture rewritten around the actual giant/model split
  (layers/encoders/trunks/routers/history/objectives/models/builders/
  _legacy/summary; network.py is now a re-export shim), plus
  cond_layout.py, checkpoint_io.py, _migration.py, data/setup_cache.py
  and giant/training/.
- Conditioning documented per axis (conditioning.particle /
  conditioning.material, each physical|embedding|onehot, freely mixed).
- Stage 2 documented with both decoders, n_sec.mode, teacher forcing,
  stage1_context and the three particle_type.target options.
- Roadmap: v0.3.0 recorded as implemented/released; WGAN and MoE routing
  as implemented but unvalidated, with the router retrain as next step.
- Analysis: run dir is <cwd>/analysis_runs/analysis_<id>, plus
  variables/reduced/runtime_estimate and analyze list/merge-one/metrics.
- Added giant model summary, configs/, and the CI-automated version and
  changelog bump.

README drift fixes only: project tree for the model/analysis/training
splits, analyze run-dir default, missing subcommands, --precision and
--stage2-stage1-context, the extras list, and two accuracy fixes
(--router configures stage 1 only; --conditioning sets two independent
axes at once).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 10:04:06 +02:00
lars 1e92902c8d Backfill CHANGELOG.md for v0.2.0-v0.3.2
CI / Format (ruff format) (push) Successful in 31s
CI / Lint (ruff check) (push) Successful in 33s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 30s
CI / Tests (push) Successful in 2m53s
CI / Bump version, tag, and update changelog on merge to master (push) Successful in 13s
The automated changelog (gitea #50) deliberately started fresh with no
backfill; this reverses that call now that it's wanted. v0.2.0-v0.3.2 are
generated from tag history via git-cliff/cliff.toml, matching the format of
existing entries. v0.3.3 was bumped but never tagged, so its commits stay
folded into the existing v0.3.4 entry. The v0.2.0 range (198 uncurated
pre-automation commits) is hand-curated to drop duplicate commits and
dev-log noise (WIP markers, incomplete-validation runs, repeated
"Apply ruff format").
2026-08-24 15:30:14 +02:00
gitea-actions a2d55e745f chore: update changelog for v0.3.9 [skip ci] 2026-08-24 12:37:57 +00:00
gitea-actions f62f12e49e chore: bump version 0.3.8 -> 0.3.9 [skip ci] 2026-08-24 12:37:56 +00:00
lars d07bac8d32 Merge pull request 'Add multi-rollout support to giant analyze (gitea #77)' (#80) from fix/issue-77 into master
CI / Lint (ruff check) (push) Successful in 33s
CI / Format (ruff format) (push) Successful in 32s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 24s
CI / Tests (push) Successful in 3m0s
CI / Bump version, tag, and update changelog on merge to master (push) Successful in 42s
Reviewed-on: #80
2026-08-24 14:33:01 +02:00
lars e90eead2af Escape LaTeX-special characters in plot titles/xlabels (gitea #81)
CI / Format (ruff format) (push) Successful in 32s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 33s
CI / Format (ruff format) (pull_request) Successful in 37s
CI / Lint (ruff check) (pull_request) Successful in 38s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Lint (ruff check) (push) Successful in 39s
CI / Type check (ty) (pull_request) Successful in 43s
CI / Tests (push) Successful in 4m52s
CI / Tests (pull_request) Successful in 4m52s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
shower_containment_depth_90/95's title contains a literal "%" (e.g.
"...(90% of deposited energy)"), which usetex reads as a comment marker
and aborts LaTeX compilation. Since render_all processes reduced JSON
files in sorted filename order, this killed every plot id sorting after
these two in the same run.

Escape title/xlabel once, centrally, in render()'s dispatch (the one
place every renderer kind draws them from before handing off to
plotstyle/matplotlib) rather than at each catalog.py call site, so any
future catalog title with a %, &, #, etc. is covered automatically.
_plot_metadata keeps using the unescaped Reduced for the gallery YAML,
since that's not LaTeX.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 14:22:50 +02:00
lars ebd3e0dc71 Add multi-rollout support to giant analyze (gitea #77)
CI / Lint (ruff check) (push) Successful in 32s
CI / Format (ruff format) (push) Successful in 30s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 35s
CI / Lint (ruff check) (pull_request) Successful in 33s
CI / Format (ruff format) (pull_request) Successful in 30s
CI / Type check (ty) (pull_request) Successful in 34s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Tests (push) Successful in 5m59s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Tests (pull_request) Successful in 4m22s
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
giant analyze compares N rollout YAMLs against one shared reference file
(all must name the same dataset, checked up front) instead of exactly one
rollout vs one reference, rendering each rollout as its own colored series
against a single reference line/panel. Series names come from a repeated
--label flag, else the YAML stem, else "rollout" for a single YAML — a
single-rollout run keeps rendering identically to before this change.

Bundle now holds a name-keyed dict of rollout sides instead of one fixed
pair, every catalog compute_partial/finalize builds a Reduced.payload
keyed the same way ("series": {name: ...}, "reference": ... as the one
distinguished non-rollout entry), and every renderer draws N series (or
N panels, for the two heatmap-shaped specs and the router/type-embedding
diagnostics, which are inherently one-matrix/one-checkpoint per rollout)
against the reference's fixed dashed-ink style.
2026-08-24 13:23:50 +02:00
gitea-actions b8f8965338 chore: update changelog for v0.3.8 [skip ci] 2026-08-24 09:43:39 +00:00
gitea-actions 81d22c1964 chore: bump version 0.3.7 -> 0.3.8 [skip ci] 2026-08-24 09:43:38 +00:00
lars 417b741484 Merge pull request 'Add giant analyze metrics plots for training progress (gitea #75)' (#78) from fix/issue-75 into master
CI / Format (ruff format) (push) Successful in 42s
CI / Lint (ruff check) (push) Successful in 44s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 39s
CI / Tests (push) Successful in 2m50s
CI / Bump version, tag, and update changelog on merge to master (push) Successful in 33s
Reviewed-on: #78
2026-08-24 11:32:34 +02:00
lars 37d73e6578 Merge branch 'master' into fix/issue-75
CI / Format (ruff format) (push) Successful in 29s
CI / Lint (ruff check) (push) Successful in 30s
CI / Sync project version with tag (push) Has been skipped
CI / Lint (ruff check) (pull_request) Successful in 29s
CI / Type check (ty) (push) Successful in 33s
CI / Format (ruff format) (pull_request) Successful in 44s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 41s
CI / Tests (push) Successful in 5m15s
CI / Tests (pull_request) Successful in 4m42s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
2026-08-24 11:32:19 +02:00
gitea-actions ff204732d7 chore: update changelog for v0.3.7 [skip ci] 2026-08-24 09:31:26 +00:00
gitea-actions 02ed4e531c chore: bump version 0.3.6 -> 0.3.7 [skip ci] 2026-08-24 09:31:25 +00:00
lars 1b6c8b33b7 Merge pull request 'Add rollout-quality distance, confusion, containment and router plots (gitea #76)' (#79) from fix/issue-76 into master
CI / Lint (ruff check) (push) Successful in 31s
CI / Format (ruff format) (push) Successful in 34s
CI / Type check (ty) (push) Successful in 37s
CI / Sync project version with tag (push) Has been skipped
CI / Tests (push) Successful in 2m45s
CI / Bump version, tag, and update changelog on merge to master (push) Successful in 42s
Reviewed-on: #79
2026-08-24 11:22:11 +02:00
lars 7560e2bff0 Fix LaTeX-unavailable skip check in analyze metrics smoke test
CI / Lint (ruff check) (push) Successful in 27s
CI / Format (ruff format) (push) Successful in 43s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 37s
CI / Lint (ruff check) (pull_request) Successful in 29s
CI / Format (ruff format) (pull_request) Successful in 28s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 43s
CI / Tests (push) Successful in 6m14s
CI / Tests (pull_request) Successful in 4m25s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
CliRunner stores an uncaught exception in result.exception, not
result.output, so the skip condition never matched and the test
failed outright on CI machines without LaTeX installed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 11:15:51 +02:00
lars ffb7c0cc2a Add rollout-quality distance, confusion, containment and router plots (gitea #76)
CI / Format (ruff format) (push) Successful in 30s
CI / Lint (ruff check) (push) Successful in 30s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 32s
CI / Lint (ruff check) (pull_request) Successful in 33s
CI / Format (ruff format) (pull_request) Successful in 31s
CI / Type check (ty) (pull_request) Successful in 35s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Tests (push) Successful in 5m51s
CI / Tests (pull_request) Successful in 5m5s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
Picks 4 of the 7 catalog additions the issue proposed (the smaller-lift
ones; 2D joint plots, PIT calibration, and the throughput/accuracy scatter
are left for follow-up issues):

- marginal_distance_summary: a var x grouping-axis KS-statistic heatmap,
  reusing the existing marginal hist1d compute and just adding a finalize —
  a single at-a-glance regression scorecard instead of N overlay plots.
- n_sec_confusion: predicted (rollout) vs true (reference) secondary count
  per event, paired by event_id since a rollout is seeded from the same
  events as its reference file. Needed a new zero-filling primitive
  (reduce.sec_count_by_event) since a plain group_by over secondary rows
  silently drops zero-secondary events.
- shower_containment_depth_{90,95}: per-event depth containing 90%/95% of
  deposited energy, derived from the same per-event depth-bin matrix the
  longitudinal profile already computes.
- router_specialization: max gate weight vs energy per side, summarizing
  router_gating's full stacked area into the one trend line the roadmap's
  MoE writeup describes (the ~60-65% ceiling), to make a future
  lambda_balance>0 retrain's effect on specialization checkable at a glance.

Both new heatmap-shaped plots (distance summary, confusion matrix) share one
new "heatmap" Reduced kind/renderer rather than two near-identical ones.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 11:12:16 +02:00
lars bdebd83c8b Add giant analyze metrics plots for training progress (gitea #75)
CI / Lint (ruff check) (push) Successful in 27s
CI / Format (ruff format) (push) Successful in 27s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 24s
CI / Lint (ruff check) (pull_request) Successful in 39s
CI / Format (ruff format) (pull_request) Successful in 34s
CI / Type check (ty) (pull_request) Successful in 38s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Tests (push) Failing after 5m55s
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Tests (pull_request) Failing after 3m52s
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
MetricsCollector writes one row per epoch to <run_dir>/metrics.csv, but
nothing read or plotted it. giant/training/plots.py reads the CSV header
dynamically (the column set varies by run: flow/ddpm vs wgan, routed vs
not) and renders loss/lr/accuracy/grad-norm/router/wgan-balance/throughput
plots with the same plotstyle conventions giant/analysis/render.py uses,
skipping any figure whose columns aren't present for a given run.

Wired up as `giant analyze metrics <run_dir>`, writing PDFs into the same
gitignored analysis_runs/ directory `analyze prep`/`submit` already use
(derive_metrics_dir mirrors derive_run_dir) rather than into the training
run directory itself.
2026-08-24 10:55:15 +02:00
gitea-actions 97f5bbf9f0 chore: update changelog for v0.3.6 [skip ci] 2026-08-24 08:02:49 +00:00
gitea-actions 060353ea4a chore: bump version 0.3.5 -> 0.3.6 [skip ci] 2026-08-24 08:02:48 +00:00
lars b3f28e98af Merge pull request 'Give CriticModel a registry-built trunk and StageModel base (gitea #57)' (#74) from fix/issue-57 into master
CI / Format (ruff format) (push) Successful in 31s
CI / Lint (ruff check) (push) Successful in 31s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 33s
CI / Tests (push) Successful in 2m44s
CI / Bump version, tag, and update changelog on merge to master (push) Successful in 41s
Reviewed-on: #74
2026-08-24 09:57:58 +02:00
lars 4b2e0ba98e Give CriticModel a registry-built trunk and StageModel base (gitea #57)
CI / Format (ruff format) (push) Successful in 33s
CI / Lint (ruff check) (push) Successful in 36s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 27s
CI / Lint (ruff check) (pull_request) Successful in 27s
CI / Format (ruff format) (pull_request) Successful in 29s
CI / Tests (push) Successful in 3m33s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Bump version, tag, and update changelog on merge to master (push) Has been skipped
CI / Type check (ty) (pull_request) Successful in 32s
CI / Tests (pull_request) Successful in 2m45s
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
CriticModel was the one stage-shaped class left out of the trunk-registry
(gitea #33), block-conditioning-registry (gitea #34), and StageModel-base
(gitea #39) refactors: it hand-rolled a plain ResBlock stack, so a
routed/FiLM/AdaLN trunk was available to every generative stage model except
the critic competing against them under WGAN-GP.

CriticModel now subclasses StageModel (reusing its cond_enc construction, and
a stage-2 context-fusion helper factored out of Stage2OneShot onto the base)
and builds its body via build_trunk (output width 1) instead of a bespoke
ResBlock loop, so trunk.type/trunk.block_conditioning now affect the critic
too. Each stage's critic inherits its own generator's trunk config rather
than a new critic_trunk config key, mirroring the existing
critic_hidden_dim/critic_n_res_blocks "0 = inherit from generator" pattern.
Router mixing (MoE) for the critic stays out of scope. Since CriticModel is
training-only and never persisted for inference, and WGAN-GP is still
unbenchmarked, its state_dict shape has no back-compat burden.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-24 09:48:26 +02:00
30 changed files with 3274 additions and 614 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
[tool.bumpversion] [tool.bumpversion]
current_version = "0.3.5" current_version = "0.3.11"
parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)" parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
serialize = ["{major}.{minor}.{patch}"] serialize = ["{major}.{minor}.{patch}"]
search = "{current_version}" search = "{current_version}"
+524 -1
View File
@@ -1,5 +1,53 @@
# Changelog # Changelog
## [0.3.11] - 2026-08-26
### Changed
- Feat(analysis): per-step secondary multiplicity plots
## [0.3.10] - 2026-08-26
### Changed
- Backfill CHANGELOG.md for v0.2.0-v0.3.2
- Docs: bring README and CLAUDE.md in line with v0.3.9
## [0.3.9] - 2026-08-24
### Added
- Add multi-rollout support to giant analyze [gitea #77](https://git.larsbogner.de/lars/giant/issues/77)
### Changed
- Escape LaTeX-special characters in plot titles/xlabels [gitea #81](https://git.larsbogner.de/lars/giant/issues/81)
## [0.3.8] - 2026-08-24
### Added
- Add giant analyze metrics plots for training progress [gitea #75](https://git.larsbogner.de/lars/giant/issues/75)
### Fixed
- Fix LaTeX-unavailable skip check in analyze metrics smoke test
## [0.3.7] - 2026-08-24
### Added
- Add rollout-quality distance, confusion, containment and router plots [gitea #76](https://git.larsbogner.de/lars/giant/issues/76)
## [0.3.6] - 2026-08-24
### Changed
- Give CriticModel a registry-built trunk and StageModel base [gitea #57](https://git.larsbogner.de/lars/giant/issues/57)
## [0.3.5] - 2026-08-24 ## [0.3.5] - 2026-08-24
### Added ### Added
@@ -31,4 +79,479 @@
- Document CI_TOKEN's write:repository scope requirement [gitea #50](https://git.larsbogner.de/lars/giant/issues/50) - Document CI_TOKEN's write:repository scope requirement [gitea #50](https://git.larsbogner.de/lars/giant/issues/50)
# Changelog ## [0.3.2] - 2026-08-17
### Added
- Add configs/baseline.toml as the kept reference model
### Fixed
- Clamp analysis histogram bins before the i32 cast, not after [gitea #61](https://git.larsbogner.de/lars/giant/issues/61)
- Clip raw predicted log_mass in decode_secondaries [gitea #54](https://git.larsbogner.de/lars/giant/issues/54)
### Changed
- Let dwarf warm-cache take --config so it can't under-warm a config's cache keys [gitea #59](https://git.larsbogner.de/lars/giant/issues/59)
- Implement n_sec.mode = "stop_token" for the AR secondary decoder [gitea #40](https://git.larsbogner.de/lars/giant/issues/40)
- Bump patch version to 0.3.2
## [0.3.1] - 2026-08-14
### Added
- Add an Objective registry for the flow/ddpm/wgan generator choice [gitea #32](https://git.larsbogner.de/lars/giant/issues/32)
### Changed
- Make trunk architecture selectable via a registry [gitea #33](https://git.larsbogner.de/lars/giant/issues/33)
- Make ResBlock's conditioning-injection mechanism selectable [gitea #34](https://git.larsbogner.de/lars/giant/issues/34)
- Make HistoryEncoder a pluggable registry, like Router/Objective [gitea #35](https://git.larsbogner.de/lars/giant/issues/35)
- Deduplicate n_sec_head/type_head MLPs into build_mlp_head [gitea #36](https://git.larsbogner.de/lars/giant/issues/36)
- Give the cond_cat/cond_cont column layout one owner [gitea #37](https://git.larsbogner.de/lars/giant/issues/37)
- Give Stage1Model/Stage2OneShot/Stage2Autoregressive a shared StageModel base [gitea #39](https://git.larsbogner.de/lars/giant/issues/39)
- Pass ConditioningAxisConfig/ParticleTypeConfig themselves instead of raw dicts [gitea #38](https://git.larsbogner.de/lars/giant/issues/38)
- Bump patch version to 0.3.1
## [0.3.0] - 2026-08-13
### Added
- Add v0.3.0 design doc: Stage-2 autoregressive redesign
- Add pytest-cov to dev deps and run coverage in CI
- Add coverage for router-center seeding, geometry batch reader, material topN cache, and setup-cache corruption paths
- Add render.py coverage: figure params, router diagnostics plots, gallery/condor glue
- Add unknown-key validation to config.toml merge (issues.md Issue 2)
- Add consumed-keys audit test (issues.md Issue 5)
### Fixed
- Fix test_render_all_run_gallery_invokes_subprocess clobbering LaTeX's own subprocess.run
### Removed
- Remove issues.md
### Changed
- Refine v0.3.0 design: defaults, deferred scope, open questions
- Document the differentiability position and its validation obligation
- V0.3.0 step 1: new nested config schema, v0.2 migration shim
- V0.3.0 step 2: network.py refactor to composable stage models
- V0.3.0 step 3: per-stage train.py trainers + pipeline.py/cli.py rewrite
- V0.3.0 step 4: type map + particle_type.target = "onehot"/"embedding"
- V0.3.0 step 5: Stage2Autoregressive (history=markov) + §11.4 grad instrumentation
- V0.3.0 step 6: sample.py/rollout.py AR generation + class->PDG decode
- V0.3.0 step 7: AttentionHistory (KV-cached) + scheduled/never teacher forcing
- V0.3.0 post-implementation audit: resolve all 9 tracked discrepancies
- Refactor train.py into giant/training/ around a metrics collector
- Silence the fork-safety warning from num_workers>0 pipeline tests
- Deduplicate giant/training/trainers.py shared per-stage logic
- Rewrite README for v0.3.0 architecture, quick start, and data columns
- Bump version to 0.3.0
- Delete docs/v0.3.0-design.md and strip all references to it
- Apply ruff format
- Downgrade coverage-report upload to actions/upload-artifact@v3
- Bump ruff line-length to 120 and reformat
- Make config dataclasses the single source of truth for DEFAULT_CONFIG
- Extract giant train/new-run's CLI override mapping into a table-driven function (issues.md Issues 3 & 4)
- Mark issues.md Issues 3 & 4 as fixed
- Extract predict/rollout's duplicated inference bootstrap into giant.checkpoint_io (issues.md Issue 5)
- Mark issues.md Issue 5 as fixed
- Unify the two v0.2->v0.3 migration surfaces (issues.md Issue 6)
- Type the data/model/training batch contracts with NamedTuples (issues.md Issue 7)
- Split giant/model/network.py into giant/model/ (issues.md Issue 8)
- Move scripts/ to giant/tools/ (issues.md Issue 9)
- Reject stage2_model.stage1_context = 'sampled' as unimplemented (issues.md Issue 1)
- Honour wgan.critic_hidden_dim/critic_n_res_blocks in build_critics [gitea #28](https://git.larsbogner.de/lars/giant/issues/28)
- Validate stage2_model.autoregressive.order in validate_config [gitea #30](https://git.larsbogner.de/lars/giant/issues/30)
- Decouple secondary-species vocabulary from conditioning.particle.emb_dim [gitea #29](https://git.larsbogner.de/lars/giant/issues/29)
- Skip router auxiliary loss compute when their lambda is 0 [gitea #31](https://git.larsbogner.de/lars/giant/issues/31)
## [0.2.0] - 2026-08-04
### Added
- Add CLAUDE.md with architecture overview and dev commands
- Add streaming data pipeline and giant CLI entry point
- Add giant predict command
- Add ROOT-to-parquet conversion script with convert dependency group
- Add post_pos as a model target via travel_dir decomposition
- Add --coord local mode to predict for raw-space prediction debugging
- Add KL divergence to marginal validation and hook it into the training loop
- Add graceful shutdown on SIGINT/SIGTERM
- Add configurable dropout to ResBlocks
- Add giant.analysis module for notebook-based model quality diagnostics
- Add lazy polars I/O and duplicate KL/constraint checks for giant.analysis
- Add ruff and ty as dev dependencies, fix lint/type findings
- Add linear warmup before cosine LR decay
- Add --batch-size auto to estimate batch size from free GPU memory
- Add hyperparameter scan
- Add --batch-size auto to predict, matching train
- Add tqdm progress bar to predict
- Add KL bar plots and sample_frac to load_predicted_local; ignore root parquet scratch files
- Add event-level shower observables to giant.analysis
- Add total length traveled per event to event observables
- Add pdg energy/length contribution pie plots
- Add export script for Tier 4 event-level/pdg-share plots
- Add mean/median deposited energy and step length plots per event
- Add export script for ETP group-update presentation plots
- Add photon edep export scripts and per-step presentation plots
- Add tooling for a versioned geant_steps dataset layout
- Add --copy mode to migrate_geant_steps.py
- Add update-manifest and create-manifest subcommands to bump_dataset_version
- Add --to flag for bump-gen/bump-schema and --gen flag for update-manifest
- Add disk usage summary to dwarf status
- Add file counts and reference tracking to dwarf status
- Add --comment option to predict, recorded in YAML sidecar
- Add energy-conservation PoC ODE-step comparison scripts
- Add autoregressive shower rollout driver
- Add fast slab lookup for the GeometryOracle, replacing knn as the default
- Add load_rollout_vs_truth to compare rollouts against held-out truth data
- Add mixture-of-experts routing prototype for Stage 1 and Stage 2
- Add ProcessRouter for physics-process-based expert gating
- Add PdgRouter for particle-type-based expert gating
- Add ComposedRouter for multi-axis MoE gating
- Add EMA weights, weight decay, step-based LR schedule, and grad-norm logging to training
- Add WGAN-GP mode as a throwaway fast-eval experiment
- Add router gating diagnostic for MoE checkpoints
- Add Gitea Actions CI pipeline
- Add configs for router energy (embedding/physical) and WGAN baseline runs
- Add opt-in Weights & Biases logging for the training loop
- Add test coverage for resolve_expert_dims
- Add regression coverage for vocab/process index-map builders
- Add dwarf warm-cache to precompute the setup-stage sidecar
- Add giant new-run to scaffold a config.toml + run dir ahead of training
- Add learnable per-expert width and shared temperature to EnergyRouter
- Add opt-in straight-through Gumbel-softmax combine weights to MoE router
- Add gumbel router configs sweeping learnable-knob combinations
- Add gumbel/learn_centers/learn_width/learn_temperature to out-dir naming
- Add bigger WGAN config (hidden_dim=512, n_blocks=6)
- Add data-integrity guards against silent NaN/Inf propagation and races
### Fixed
- Fix column names to match actual parquet schema
- Fix installed torch version to be compatible with cuda drivers
- Fix miniCaloSim link in README
- Fix giant.analysis import after Phase 2 dataset API changes
- Fix silent failure modes surfaced by extensive code review
- Fix ruff, ty, and pytest failures; apply ruff format
- Clamp n_sec classification label to K_MAX
- Fix rollout edep mismatch and add truth overlay to Tier 4 observables
- Fix crashes in physical-property conditioning edge cases
- Fix router experts silently ignoring --hidden-dim/--n-blocks
- Fix conditioning="physical" so it can actually generalize past training vocab
- Fix training-loop checkpoint/resume and WGAN bugs
- Fix stale-partial reuse and n_chunks mismatch in analysis condor pipeline
- Fix CLI/tooling robustness gaps and dedupe the Conditioning enum
- Fix test_write_submit_requires_synced_venv for active-venv resolution
### Removed
- Remove scripts/train.py in favor of the giant train CLI
- Drop orphaned child tracks instead of nulling secondary targets
### Changed
- Initial commit: giant surrogate model with two-phase roadmap in README
- Implement Phase 1: full data pipeline, model, training, and config support
- Handle material column as string type
- Rename pre_energy/post_energy columns to pre_E/post_E
- Rename direction columns from pre_dir_x/y/z to pre_dx/dy/dz
- Batch StreamingStepsDataset internally instead of per-row collate
- Dedup training pipeline, add seeding/resume and per-epoch metrics logging
- Split torch into cpu/cuda extras and pin dependency version bounds
- Apply ruff format and document lint/type tooling in CLAUDE.md
- Update README to match current architecture and tooling
- Make sampler step count configurable for validation
- Calibrate auto batch size separately for inference vs training
- Skip rows with unknown PDG codes during predict
- Buffer predict rows across row-group boundaries before inference
- Export plots for knowledge base
- Rework validation notebook with markdown sections and Tier 4 plots
- Allow steps_to_parquet.py to accept multiple ROOT input files
- Encode edep/secondary/post energy as a conservation-constrained simplex
- Expose dataset/conversion scripts as uv entry points
- Restrict holdout overlap check to holdout vs dev/full only
- Route predict output to UUID-named parquet with YAML reference sidecar
- Implement Phase 2: secondary particle prediction
- Unify dataset/tooling scripts into a single `dwarf` Typer CLI
- Fold --to/--gen dataset-versioning flags into the dwarf CLI
- Prefix default train output dir with current date
- Color-code dwarf status output by tree level
- Show VERSIONS.md reason extracts in dwarf status
- Wire up predict CLI to load and run the Stage-2 sec_decoder
- Wire up n_sec/species/energy-fraction validation for Stage 2
- Detach Stage-2 type-embedding target to stop self-referential collapse
- Weight Stage-2 secondary loss equally between direction and type-embedding dims
- Recalibrate batch-size estimate for the post-Phase-2 model size
- Update CLAUDE.md and README for the implemented Phase 2 model
- Error on missing secondary lists instead of silently zeroing Stage-2 targets
- Derive a unique per-job seed for minicalosim shard generation
- Rescale secondary energies to exactly consume the e_sec budget
- Support --energy-gev in dwarf make-root for the new minicalosim energy arg
- Stream giant rollout output instead of buffering the whole run
- Scale auto batch-size estimate by MoE expert count during training
- Rewrite analysis module as a lean, fully-streaming pipeline
- Reimplement rollout-vs-truth comparison on the streaming analysis module
- Condition on material/particle physical properties instead of learned embeddings
- Ignore the scratchpad working directory
- Quote the on: key in the CI workflow
- Split CI lint stage into parallel jobs
- Rewrite analysis as streaming rollout-vs-reference plotting pipeline
- Analyze: drive prep/submit from the rollout YAML sidecar
- Analyze: show model/training params on rendered figures
- Deps: install plotstyle from git.larsbogner.de package index
- Analyze: drop stale ty:ignore on plotstyle import
- Test: replace prep(**_CTX) splat with a typed _prep helper
- Analyze: add MoE router gating/share diagnostic plots
- Chore: remove stray CUDA sanity script and stale Phase 2 planning doc
- Docs: document compute environment, WGAN/MoE status, and condor-gpu-train-rollout
- Analyze: normalize pdg dtype in open_side to fix rollout/reference concat
- Analyze: chunk per-plot aggregation across HTCondor jobs
- Analyze: expose bin/pdg options on `analyze submit`
- Analyze: estimate per-job HTCondor walltime from chunk row count
- Analyze: run condor compute jobs via .venv/bin/giant, not uv run
- Analyze: default condor docker image to alma9-gridjob
- Analyze: raise default condor job memory request to 8192 MB
- Analyze: recalibrate condor walltime model from real cluster timings
- Transforms: pad legacy cond normalizers for pre-physical-conditioning checkpoints
- Analyze: default run directory to <repo>/analysis_runs, gitignored
- Docs: record first MoE router rollout benchmark result in the roadmap
- Router: seed EnergyRouter centers from data quantiles instead of a fixed linspace
- Docs: note the EnergyRouter centers_init fix in the roadmap
- Analyze: thread full model/training/rollout/dataset params to plots
- Ci: share one uv sync across jobs, gate tests on lint+type-check, sync tag/version on release tags
- Ci: replace unsupported artifact sharing with a bind-mounted uv cache
- Ci: stop setup-uv from overriding UV_CACHE_DIR
- Ci: re-pin UV_CACHE_DIR after setup-uv, which exports its own value regardless of enable-cache
- Ci: set UV_LINK_MODE=copy to silence the cross-filesystem hardlink warning
- Log batch-level metrics to W&B, not just per-epoch summaries
- Log router health, WGAN grad-norm split, n_sec accuracy, GPU/throughput to W&B
- Persist global_step across --resume so W&B step stays monotonic
- Timestamp default checkpoint dir to avoid W&B run-id collisions
- Skip empty-slice mean/std in sec phys validation print
- Speed up giant train's setup stage
- Speed up _WelfordAccumulator's per-chunk update
- Make default checkpoint out_dir name reflect only non-default hyperparams
- Cache giant train's setup stage in a sidecar file
- Pass --seed through to the train/val event split
- Offset event_id per file to avoid cross-file collisions
- Store a quantile grid instead of a raw reservoir sample in the setup cache
- Scope wandb run config to only-active hyperparameters
- Resolve giant condor wrapper from the active venv, not a hardcoded path
- Bump version to 0.2.0
+57 -29
View File
@@ -7,18 +7,20 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
```bash ```bash
uv sync --extra cpu # install dependencies with CPU-only torch (standard/default) uv sync --extra cpu # install dependencies with CPU-only torch (standard/default)
uv sync --extra cuda # install dependencies with CUDA 11.8 torch uv sync --extra cuda # install dependencies with CUDA 11.8 torch
uv sync --extra cpu --extra dev # add dev extras (pytest, etc.) uv sync --extra cpu --extra dev # add dev extras (pytest, ruff, ty, bump-my-version, git-cliff, + all runtime extras)
uv sync --extra cpu --extra geometry # add scikit-learn for the geometry oracle (giant rollout) uv sync --extra cpu --extra geometry # add scikit-learn for the geometry oracle (giant rollout)
pytest # run tests pytest # run tests
giant new-run --hidden-dim 512 --lr 3e-4 # scaffold a config.toml + run dir ahead of training giant new-run --hidden-dim 512 --lr 3e-4 # scaffold a config.toml + run dir ahead of training
giant train path/to/steps.parquet --mode flow # train (flow matching) giant train path/to/steps.parquet # train (defaults: stage 1 flow, stage 2 wgan + autoregressive)
giant train path/to/steps.parquet --mode ddpm # train (DDPM baseline) giant train path/to/steps.parquet --mode flow # set both stages' generative objective at once
giant train path/to/steps.parquet --mode wgan # train (WGAN-GP, single-pass eval; implemented, not yet tested) giant train path/to/steps.parquet --stage1-generator flow --stage2-generator wgan # per-stage override
giant train path/to/steps.parquet --router --router-type energy # MoE routing trunk (implemented; first rollout benchmark failed with lambda_balance=0, retrain needed — see Roadmap) giant train path/to/steps.parquet --router --router-type energy # MoE routing trunk (see Roadmap for status)
giant model summary --config config.toml # build-only: parameter counts + which config keys actually bite
giant predict path/to/steps.parquet --checkpoint ckpt/best.pt # per-step predictions giant predict path/to/steps.parquet --checkpoint ckpt/best.pt # per-step predictions
giant rollout path/to/steps.parquet --checkpoint ckpt/best.pt --geometry oracle.pkl # full showers giant rollout path/to/steps.parquet --checkpoint ckpt/best.pt --geometry oracle.pkl # full showers
giant analyze submit rollout.yaml --accounting-group cms # parallel rollout-vs-reference analysis on HTCondor giant analyze submit rollout.yaml --accounting-group cms # parallel rollout-vs-reference analysis on HTCondor
giant analyze render <run_dir> --gallery # render PDFs + HTML gallery (run_dir from prep/submit) giant analyze render <run_dir> --gallery # render PDFs + HTML gallery (run_dir from prep/submit)
giant analyze metrics <train_run_dir> # training-progress plots from metrics.csv
dwarf --help # dataset/tooling CLI: convert, migrate, bump-gen, dwarf --help # dataset/tooling CLI: convert, migrate, bump-gen,
# bump-schema, status, update-manifest, create-manifest, # bump-schema, status, update-manifest, create-manifest,
# make-root, build-geometry-oracle, warm-cache, hparam-scan # make-root, build-geometry-oracle, warm-cache, hparam-scan
@@ -27,6 +29,8 @@ dwarf --help # dataset/tooling CLI: convert,
`cpu` and `cuda` are mutually exclusive — pick one to select the torch build (pinned to 2.3.x; newer torch requires newer NVIDIA drivers). Plain `uv sync` with no extra will not install torch at all; uv has no concept of a "default extra", so `--extra cpu` should always be included unless you need GPU support. `cpu` and `cuda` are mutually exclusive — pick one to select the torch build (pinned to 2.3.x; newer torch requires newer NVIDIA drivers). Plain `uv sync` with no extra will not install torch at all; uv has no concept of a "default extra", so `--extra cpu` should always be included unless you need GPU support.
`configs/` holds kept reference configs (`baseline.toml`, `default.toml`, the router/WGAN scan configs) — pass them with `--config`.
### Lint and type checking ### Lint and type checking
```bash ```bash
@@ -37,6 +41,10 @@ uv run ty check . # type check
Part of the `dev` extra. Run these periodically (not just at commit time) to catch drift early. Part of the `dev` extra. Run these periodically (not just at commit time) to catch drift early.
### Release tooling
Merges to `master` auto-bump the patch version, tag, and update `CHANGELOG.md` via the Gitea workflow in `.gitea/workflows/ci.yml` (bump-my-version + git-cliff). Don't hand-edit the version in `pyproject.toml` or write changelog entries by hand.
## Compute environment ## Compute environment
Work on this repo happens across three kinds of machine: Work on this repo happens across three kinds of machine:
@@ -47,50 +55,70 @@ Work on this repo happens across three kinds of machine:
## Architecture ## Architecture
GIANT is a conditional generative surrogate for the Geant4 step function. It replaces the stochastic physics engine: given a pre-step particle state (conditioning), it samples a post-step outcome — now including the variable-length list of secondary particles the step produces (Phase 2, see Roadmap). GIANT is a conditional generative surrogate for the Geant4 step function. It replaces the stochastic physics engine: given a pre-step particle state (conditioning), it samples a post-step outcome — including the variable-length list of secondary particles the step produces.
**Data pipeline** (`giant/data/`): parquet files from miniCaloSim are loaded into numpy arrays (`loader.py`), then log-transformed and rotated into a local coordinate frame where `pre_dir = ẑ` (`transforms.py`), before being wrapped in a PyTorch `Dataset` (`dataset.py`). Train/val split is by `event_id` to avoid leaking correlated steps from the same shower. **Data pipeline** (`giant/data/`): parquet files from miniCaloSim are loaded into numpy arrays (`loader.py`), then log-transformed and rotated into a local coordinate frame where `pre_dir = ẑ` (`transforms.py`), before being wrapped in a PyTorch `Dataset` (`dataset.py`, streaming variant included). Train/val split is by `event_id` (`--seed`-controlled) to avoid leaking correlated steps from the same shower. Loading a directory or `.manifest` of several parquet files offsets each file's `event_id`s by a per-file stride so ids stay globally unique. `setup_cache.py` persists the pre-epoch setup scan (vocab maps, event split, process maps, normalizer stats) as a sidecar so repeated runs over the same `data` path don't rescan (`--cache-setup`/`--rebuild-setup-cache`, precomputable with `dwarf warm-cache --config ...`).
**Stage-1 output space (9D, `giant/constants.py:LOCAL_TARGET_NAMES`):** `log_step_length`, two additive-log-ratio (ALR) coordinates `edep_logit`/`sec_logit` of a **deposit / secondary / post-energy simplex**, `post_dir` (post-scattering momentum direction, unit vector in the local frame), and `travel_dir` (direction of `post_pos - pre_pos`, unit vector in the local frame). The energy simplex decodes via softmax over `[edep_logit, sec_logit, 0]` × `pre_E` so `edep + e_sec + post_E == pre_E` holds by construction — energy conservation is architectural, not learned (see `energy_simplex_decode`). `post_pos` is not a raw target — it's reconstructed at inference as `pre_pos + step_length * world_frame(travel_dir)`, since `step_length` already encodes that displacement's magnitude and duplicating it would let the two become inconsistent. **Stage-1 output space (9D, `giant/constants.py:LOCAL_TARGET_NAMES`):** `log_step_length`, two additive-log-ratio (ALR) coordinates `edep_logit`/`sec_logit` of a **deposit / secondary / post-energy simplex**, `post_dir` (post-scattering momentum direction, unit vector in the local frame), and `travel_dir` (direction of `post_pos - pre_pos`, unit vector in the local frame). The energy simplex decodes via softmax over `[edep_logit, sec_logit, 0]` × `pre_E` so `edep + e_sec + post_E == pre_E` holds by construction — energy conservation is architectural, not learned (see `energy_simplex_decode`). `post_pos` is not a raw target — it's reconstructed at inference as `pre_pos + step_length * world_frame(travel_dir)`, since `step_length` already encodes that displacement's magnitude and duplicating it would let the two become inconsistent.
**Conditioning vector (15D continuous, `COND_DIM`):** pre-step position, log(pre-energy), pre-step direction, layer ID (`COND_DIM_BASE=8`) — plus, since particle/material physical-property conditioning (`model.conditioning`, see below), 7 more columns: particle `log(mass)`/`charge` (`PARTICLE_PHYS_DIM=2`, `giant/particles.py`) and material `Z_eff`/`A_eff`/`log(density)`/`log(X0)`/`log(λ_int)` (`MATERIAL_PHYS_DIM=5`, `giant/materials.py`). `n_sec` and `e_sec` are **not conditioning inputs** (that was Phase 1 / the energy-conservation PoC); the model predicts them. **Conditioning vector (15D continuous, `COND_DIM`):** pre-step position, log(pre-energy), pre-step direction, layer ID (`COND_DIM_BASE=8`) — plus 7 physical-property columns: particle `log(mass)`/`charge` (`PARTICLE_PHYS_DIM=2`, `giant/particles.py`) and material `Z_eff`/`A_eff`/`log(density)`/`log(X0)`/`log(λ_int)` (`MATERIAL_PHYS_DIM=5`, `giant/materials.py`). `n_sec` and `e_sec` are **not conditioning inputs** — the model predicts them. `giant/cond_layout.py` is the single source of truth for the `cond_cont`/`cond_cat` column layout shared by `giant.data.transforms`, `giant.model.encoders`, and `giant.model.routers`.
`ConditionEncoder`/`SecondaryConditionEncoder` (`giant/model/network.py`) support two mutually exclusive `conditioning` modes, selected per-checkpoint (`model_config["conditioning"]`, defaulting to `"embedding"` for old checkpoints without the key, `"physical"` for new `giant train` runs — see `--conditioning`): `ConditionEncoder` (`giant/model/encoders.py`) configures the particle and material identity axes **independently** (`conditioning.particle` / `conditioning.material`, each a `ConditioningAxisConfig` with `type`/`emb_dim`/`n_layers`), so they may mix freely. Three per-axis modes:
- **`"embedding"`** (original Phase 2 design): a learned `nn.Embedding` per PDG code / material name, indexed by a dataset-scoped dense vocab (`pdg_map`/`mat_map`). Memorizes the training menu. - **`"physical"`** (default): the axis's raw physical properties routed through a small MLP — computable for any PDG code / material name, which is what lets the surrogate generalize beyond the training menu. `giant/particles.py` decodes nuclear/ion PDG codes (the `10LZZZAAAI` scheme) via the scikit-HEP `particle` package with a Z/A-digit-decode fallback for isomer codes the package's ground-state-only table misses. `giant/materials.py` ships real Geant4-11.4.1-derived values for every material the detector geometry actually produces; the sole exception is `G4_LYSO` (not a stock Geant4 NIST material, never actually constructed by the geometry — see the module docstring), which stays `MaterialProperties(None, ...)` and raises loudly (`MaterialPropertiesNotFilledError`) rather than silently defaulting.
- **`"physical"`** (default): the 7 physical-property columns above are each routed through a small MLP (`particle_mlp`/`material_mlp`) to the same `emb_dim` width the embedding tables would have produced — a drop-in replacement computable for any PDG code / material name, not just ones seen in training, which is what lets the surrogate generalize to a held-out material or species. `giant/particles.py` decodes nuclear/ion PDG codes (the `10LZZZAAAI` scheme) via the scikit-HEP `particle` package with a Z/A-digit-decode fallback for isomer codes the package's ground-state-only table misses. `giant/materials.py` ships real Geant4-11.4.1-derived `z_eff`/`a_eff`/`density`/`x0`/`lambda_int` values for every material the detector geometry actually produces; the sole exception is `G4_LYSO` (not a stock Geant4 NIST material, never actually constructed by the geometry — see the module docstring), which stays `MaterialProperties(None, ...)` and raises loudly (`MaterialPropertiesNotFilledError`) rather than silently defaulting if it's ever requested. - **`"embedding"`**: a learned `nn.Embedding` per PDG code / material name, indexed by a dataset-scoped dense vocab. Memorizes the training menu; the generalization-comparison baseline, and the only mode compatible with `stage2_model.particle_type.target = "embedding"`.
- **`"onehot"`**: a fixed, unlearned vector over the top `emb_dim - 1` codes by training-set count plus one "other" bin. Not a reparameterization of `"embedding"` — the vocabulary cap is the real difference.
**Model** (`giant/model/network.py`): a two-stage model, both checkpointed together. `conditioning.share_stages` decides whether the two stages get one shared encoder instance or two identically-configured independent ones.
- **Stage 1 — `DenoisingMLP`:** `ResBlock` stack with a `SinusoidalEmbedding` for the flow/diffusion time variable and a `ConditionEncoder` fusing the conditioning. Predicts the 9D primary vector field, plus an `n_sec_head` classifier over `{0..K_MAX}` (`K_MAX=15`) that runs on the condition encoding alone (no diffusion noise), callable via `predict_n_sec`.
- **Stage 2 — `SecondaryDecoder`:** a second flow-matching net (`SecondaryConditionEncoder` fuses the pre-step conditioning with the Stage-1 outcome) that generates all `K_MAX` secondary slots at once. Each slot is `(stick-breaking energy logit, local-frame direction 3D, log-mass, charge)` = `SEC_SLOT_DIM=6`, ordered by descending energy; slots beyond the predicted `n_sec` are masked. Secondary energies are a **stick-breaking partition of the `e_sec` budget** from Stage 1 (they sum to it), so the whole chain conserves energy. A secondary's mass/charge are regressed directly against a fixed physics-derived target (its ground-truth PDG code's `giant.particles.particle_mass_charge`) — not a learned/moving embedding target, so nothing needs detaching. **No snapping at inference**: the predicted (mass, charge) are used as-is as the secondary's physical identity, including for its own future conditioning if it goes on to take further steps in a rollout. A separate, reporting-only nearest-known-PDG lookup (`giant.particles.nearest_known_pdg`) is used purely to populate a nominal `pdg` label for output rows / `"embedding"`-mode fallback conditioning — it never feeds back into the model.
`schedule.py` provides both a `CosineSchedule` for DDPM and the flow matching loss utilities (Lipman et al. 2022 conditional flow matching). **Model** (`giant/model/`, both stages checkpointed together). `network.py` is only a re-export shim now; the real code is split by concern:
- `layers.py``ResBlock`/`AdaLNResBlock` + `BLOCK_REGISTRY` (conditioning-injection mechanism is selectable), `SinusoidalEmbedding`, `ContextAdapter`, `build_mlp_head`.
- `encoders.py``ConditionEncoder` (above).
- `trunks.py``TRUNK_REGISTRY`/`build_trunk`: everything downstream of the fused conditioning vector, as a registrable expert *body* (`resmlp` default, plus a `none` variant). `RoutedTrunk` builds `router.n_experts` instances of whichever body is named, so mixing is orthogonal to which body is mixed.
- `routers.py``Router` base + `ROUTER_REGISTRY`: `energy`/`pdg`/`process`/`composed`/`none`. Soft-mixed at train time, **top-1 dispatched at eval time** (each row runs exactly one small expert), which is the actual inference-speed win. `EnergyRouter`/`PdgRouter` gate on a quantity known at inference; `ProcessRouter` runs its own small classifier (process isn't known upfront); `ComposedRouter` gates jointly over outer-product expert cells via repeated `--router-axis "type:key=val,..."`. The `--router*`/`--n-experts` CLI flags target `stage1_model.router` only; stage 2's router is config-file-only (`stage2_model.router`). `EnergyRouter` accepts `centers_init`, which `giant/pipeline.py` auto-populates from real data quantiles via a reservoir sample collected during the normalizer-fitting pass.
- `history.py``HISTORY_REGISTRY`/`build_history`: `markov` (previous token only), `attention` (causal self-attention, KV-cached at inference via `init_cache`/`step`), `none`. Stage-2 autoregressive only.
- `objectives.py``Objective` base + registry for `flow`/`ddpm`/`wgan`: answers in one place whether a stage needs a time embedding, is adversarial, folds the secondary type slice into its trunk output, what its trunk input is, and which loss it trains against.
- `models.py` — the composed stage models: `Stage1Model`, `Stage2OneShot`, `Stage2Autoregressive`, `CriticModel`, all on a shared `StageModel` base.
- `builders.py``build_models`/`build_critics`, assembling the above from a config dict.
- `schedule.py` (`CosineSchedule` for DDPM + conditional-flow-matching losses), `wgan.py` (gradient penalty / critic / generator losses, Gulrajani et al. 2017), `summary.py` (`giant model summary`), `_legacy.py` (v0.2 checkpoint migration).
**Samplers** (`giant/sample.py`): DDPM, DDIM, and flow matching (ODE integration, ~10 steps). Flow matching is the primary mode. **Stage 1 — primary step.** Trunk (routed or not) over the fused conditioning, plus a `SinusoidalEmbedding` of the flow/diffusion time for non-adversarial objectives, predicting the 9D vector field. An `n_sec` classifier head over `{0..k_max}` runs on the condition encoding alone; `stage2_model.n_sec.owner` decides whether it lives on stage 1 (v0.2 checkpoints) or stage 2 (default).
**WGAN-GP mode (`--mode wgan`, implemented, not yet tested):** a throwaway fast-eval alternative to the flow/DDPM samplers above — single forward pass instead of ~10 ODE steps. Dedicated noise-conditioned generators (`WGANGenerator`/`WGANSecondaryGenerator`, `giant/model/network.py`) stand in for `DenoisingMLP`/`SecondaryDecoder`, trained against `Critic`/`SecondaryCritic` discriminators with the gradient-penalty loss in `giant/model/wgan.py` (Gulrajani et al. 2017); `sample_wgan` (`giant/sample.py`) does the single-pass draw at inference. Not yet validated against the flow-matching baseline. **Stage 2 — secondaries.** Conditioned on the pre-step state plus a projected stage-1 outcome (`stage2_model.context_dim`; `stage1_context` selects ground-truth vs sampled context, annealable via `ctx_p_start`/`ctx_p_end`). Two decoders (`stage2_model.decoder`):
- **`autoregressive`** (default): one secondary at a time in descending-energy order, each token conditioned on a `HistoryEncoder` summary of prior tokens, with teacher forcing (`always`/`scheduled`/`never`, `tf_p_start`/`tf_p_end`). `n_sec.mode = "stop_token"` lets the length be emitted by the sequence itself instead of the classifier head.
- **`one_shot`**: all `k_max` slots in one pass, masked past the predicted `n_sec` (the v0.2 behaviour).
**MoE routing trunk (`--router`, implemented; first rollout benchmark shows the experts don't specialize — see Roadmap):** an alternative to `DenoisingMLP`'s monolithic `ResBlock` trunk — a `Router` (`giant/model/network.py`, `ROUTER_REGISTRY`/`build_router`) gates between small per-expert `ResBlock` stacks (`Expert`), soft-mixed over all experts at train time but **top-1 dispatched at eval time** (each row runs exactly one small expert), which is the actual inference-speed win. Router types gate on different conditioning axes: `EnergyRouter`/`PdgRouter` read a quantity already known at inference time, `ProcessRouter` runs its own small classifier over pre-step conditioning (since process isn't known upfront); `ComposedRouter` gates jointly over multiple axes (outer-product expert cells) via repeated `--router-axis "type:key=val,..."` flags. Config lives under `model.router` (`giant/config.py`), deep-merged one level so `router.enabled` alone doesn't drop the rest of the defaults. Secondary energies are a **stick-breaking partition of the `e_sec` budget** from Stage 1 (they sum to it), so the whole chain conserves energy. Particle identity is set by `stage2_model.particle_type.target`: `"onehot"` (default — categorical over the top `n_classes - 1` PDG codes by training count plus "other", with configurable `other_policy` and `class_weighting`), `"physical"` (continuous `(log-mass, charge)` regressed against `giant.particles.particle_mass_charge`), or `"embedding"` (nearest-row snap into the conditioning embedding table; requires `conditioning.particle.type = "embedding"`).
**Validation** (`giant/validate.py`): step-level marginal comparisons. **Samplers** (`giant/sample.py`): DDPM, DDIM, flow matching (ODE integration, ~10 steps), and single-pass WGAN, plus the stage-2 secondary sampling loop (one-shot and autoregressive).
**Analysis** (`giant/analysis/`, `giant analyze` CLI): a lean, streaming rollout-vs-reference plotting pipeline that compares one autoregressive `giant rollout` (for a given checkpoint) against a held-out miniCaloSim reference steps file, and produces publication-styled PDFs assembled into an HTML gallery. It exploits the fact that rollout output and a raw reference file share a world-frame physical column subset under identical names (`pre_*`/`post_*`/`edep`/`step_length`/`pdg`/`material`/`event_id`), so no ALR/local-frame decode is needed — everything is world-frame mm/MeV. Structure: `sources.py` (canonical LazyFrames + synthetic-termination-row filtering + the secondary view, which is `generation>0 & step_no==0` rollout tracks vs exploded `sec_*_list` reference columns), `reduce.py` (the streaming primitives — a single `hist1d` `group_by([group,bin]).len()` pass, per-event scalars, edep-weighted depth/transverse profiles, species share, leakage), `grouping.py`/`context.py` (fixed bin edges + energy-quantile/pdg/material group sets resolved once by `prep` into `shared.json`, so every compute job is one pass with no range scan), `catalog.py` (the declarative `PlotSpec` registry — marginals × {overall,energy,pdg,material}, per-event totals, shower profiles, species/leakage, secondaries), and `render.py` (the only module importing ETPlot's `plotstyle`/LaTeX; dispatches on `Reduced.kind`, writes PDFs + `metadata.yaml`). **Input is a `giant rollout` YAML sidecar** (`condor.py:load_rollout_yaml`): its `output`/`dataset` keys name the rollout parquet and the seed file (= the reference truth), and the rest of the YAML (checkpoint, geometry oracle, cutoffs) flows into each plot's gallery metadata. `prep` derives its own **run directory** next to the rollout parquet (`<...>/analysis_<id>/`) holding `shared.json`, `run_meta.json`, `reduced_partial/`, `reduced/`, `plots/`. **Compute/merge/render split:** `giant analyze submit rollout.yaml --chunks N` runs `prep` (recording the run's chunk count `N` in `run_meta.json`) then submits one HTCondor job per (plot, chunk) pair (`compute-one --id --chunk --run-dir`, polars/numpy only — no LaTeX on workers), each streaming over an `event_id`-disjoint slice (`event_id % N == chunk`) and writing a small `reduced_partial/<id>__<chunk>.json`; every `PlotSpec` (`catalog.py`) splits into a `compute_partial`/`finalize` pair so a plot's chunks can be summed/concatenated back together correctly (`chunkable=False` specs — the router diagnostics, already bounded/subsampled — always run as a single chunk regardless of `N`). The local `giant analyze render <run_dir>` first joins every plot's chunk partials into `reduced/<id>.json` (`merge_all`, a no-op join when `N=1`), then turns those into the styled PDF/gallery tree. See `giant/analysis/__init__.py`. **Training** (`giant/training/`): `loop.py` (epoch loop, graceful shutdown, best-checkpoint selection), `trainers.py` (`StageSpec` + per-stage flow/ddpm and WGAN-GP trainers, and the `MetricSpec` declarations that define `metrics.csv`'s columns), `stage2_inputs.py` (ground-truth stage-2 targets + teacher-forcing inputs), `metrics.py` (`MetricsCollector`: `metrics.csv`, W&B logging, progress/summary), `checkpoint.py`, `amp.py` (`train.precision = fp32|bf16` autocast), `plots.py` (`giant analyze metrics`). Per-stage `init_from`/`freeze` lets one stage be retrained against a fixed, known-good other stage while still producing a complete rollout-capable checkpoint.
**Shower rollout** (`giant/rollout.py`, `giant rollout` CLI): autoregressively steps the two-stage model into a full shower — each primary post-step becomes the next pre-step, secondaries are pushed as new tracks, and per-step `material`/`layer_id` come from a `GeometryOracle` (`giant/geometry.py`, built via `dwarf build-geometry-oracle`) that learns position → (material, layer_id) from data and flags detector escape by nearest-neighbour distance. Tracks terminate on energy cutoff, per-track max steps, escape, or natural end; energy is deposited locally on every stop except escape (leakage), so showers conserve energy by construction. **Config** (`giant/config.py`): frozen dataclasses are the single source of truth; `DEFAULT_CONFIG` is *generated* from `GiantConfig().to_dict()` rather than hand-maintained. Blocks: `[conditioning]`, `[stage1_model]`, `[stage2_model]`, `[train]`, `[meta]`. Unknown keys are rejected on merge (with a did-you-mean suggestion), and `tests/test_config_consumed_keys.py` audits that every key is actually read somewhere.
**Validation** (`giant/validate.py`): step-level marginal + KL-divergence comparisons during training (`--validate-every`).
**Analysis** (`giant/analysis/`, `giant analyze` CLI): a lean, streaming rollout-vs-reference plotting pipeline that compares one or more autoregressive `giant rollout` runs against a single held-out miniCaloSim reference steps file shared by all of them, and produces publication-styled PDFs assembled into an HTML gallery — one distinctly colored series per rollout, one reference line/panel. It exploits the fact that rollout output and a raw reference file share a world-frame physical column subset under identical names (`pre_*`/`post_*`/`edep`/`step_length`/`pdg`/`material`/`event_id`), so no ALR/local-frame decode is needed — everything is world-frame mm/MeV. Structure: `sources.py` (canonical LazyFrames + `RolloutSpec`/`Side` — a rollout's opened frames + per-checkpoint diagnostic inputs — + synthetic-termination-row filtering + the secondary view, which is `generation>0 & step_no==0` rollout tracks vs exploded `sec_*_list` reference columns), `variables.py` (the per-step value expressions shared by range sizing and the plot registry), `reduce.py` (the streaming primitives — a single `hist1d` `group_by([group,bin]).len()` pass, per-event scalars, edep-weighted depth/transverse profiles, species share, leakage), `grouping.py`/`context.py` (fixed bin edges + energy-quantile/pdg/material group sets resolved once by `prep` into `shared.json` over the union of the reference and every rollout, so every compute job is one pass with no range scan), `reduced.py` (`Partial`/`Reduced` — the compact self-describing JSON a compute job emits), `catalog.py` (the declarative `PlotSpec` registry — marginals × {overall,energy,pdg,material}, per-event totals, shower profiles/containment, species/leakage, secondaries, distance summaries, router and type-embedding diagnostics; `giant analyze list` prints every id), `runtime_estimate.py` (per-(plot, chunk) walltime estimates for the submit description), and `render.py` (the only module importing ETPlot's `plotstyle`/LaTeX; dispatches on `Reduced.kind`, writes PDFs + `metadata.yaml`; each rollout gets a stable `ps.get_color(i)` slot by its position in `series`, the reference always draws in one fixed dashed-ink style). `Bundle.rollouts` is a name-keyed dict of `Side`, and every `compute_partial`/`finalize` builds a `Reduced.payload["series"]` dict keyed the same way, with `payload["reference"]` as the one distinguished non-rollout entry. The heatmap-shaped specs (`marginal_distance_summary`, `sec_count_per_step_by_species` — the latter also drawing the reference as its own panel) and the checkpoint-bound diagnostics (`router_gating.py`, `type_embedding_distance.py`) are inherently one-matrix/one-checkpoint per rollout, so they render as one panel per rollout instead of one line/bar per rollout.
**Input is one or more `giant rollout` YAML sidecars** (`condor.py:load_rollout_yamls`): each YAML's `output`/`dataset` keys name its rollout parquet and seed file (= the reference truth); every supplied YAML must resolve to the same `dataset`, checked up front with a clear error otherwise (the premise is "N candidates vs one ground truth"). Each rollout's series name comes from a repeated `--label` CLI flag, else the YAML stem (N>1), else `"rollout"` (a single YAML). `prep` creates a **run directory** (`<cwd>/analysis_runs/analysis_<id>/` by default, `--run-dir` to override) holding `shared.json`, `run_meta.json` (`RunMeta.rollouts: list[{name,path,plot_meta}]`, insertion order = CLI order = every plot's series order), `reduced_partial/`, `reduced/`, `plots/`. **Compute/merge/render split:** `giant analyze submit a.yaml [b.yaml ...] --chunks N` runs `prep` (recording `N` in `run_meta.json`) then submits one HTCondor job per (plot, chunk) pair (`compute-one --id --chunk --run-dir`, polars/numpy only — no LaTeX on workers), each streaming over an `event_id`-disjoint slice (`event_id % N == chunk`) of the reference **and every rollout** and writing a small `reduced_partial/<id>__<chunk>.json`; every `PlotSpec` splits into a `compute_partial`/`finalize` pair so chunks can be summed/concatenated back per rollout (`chunkable=False` specs — the checkpoint-bound diagnostics, already bounded/subsampled — always run as a single chunk). The local `giant analyze render <run_dir>` first joins every plot's chunk partials into `reduced/<id>.json` (`merge_all`, a no-op join when `N=1`; `merge-one` does a single plot for debugging), then turns those into the styled PDF/gallery tree. `giant analyze metrics <train_run_dir>` is a separate, unrelated entry point: training-progress plots straight from a run's `metrics.csv`.
**Shower rollout** (`giant/rollout.py`, `giant rollout` CLI): autoregressively steps the two-stage model into a full shower, advancing tracks breadth-first (every sweep steps all active tracks once, in `batch_size` chunks, so many tracks share each forward pass). Each primary post-step becomes the next pre-step, secondaries are pushed as new tracks, and per-step `material`/`layer_id` come from a `GeometryOracle` (`giant/geometry.py`, built via `dwarf build-geometry-oracle`) that learns position → (material, layer_id) from data and flags detector escape by nearest-neighbour distance. Tracks terminate on one of the `TERM_*` reasons in `constants.py` (energy cutoff, max steps, escape, natural end, unknown pdg, max tracks); energy is deposited locally on every stop except escape (leakage), so showers conserve energy by construction. `giant/checkpoint_io.py` is the shared checkpoint → ready-to-run-models path used by both `predict` and `rollout`.
## Roadmap ## Roadmap
**Phase 1 (done):** number of secondaries and their total energy were conditioning inputs; the model predicted only the 9D primary post-step (energy-conservation PoC). **Phase 1 (done):** number of secondaries and their total energy were conditioning inputs; the model predicted only the 9D primary post-step (energy-conservation PoC).
**Phase 2 (implemented — baseline):** the two-stage model above jointly predicts `n_sec`, the energy simplex (`e_sec` falls out of it), and each secondary's energy/direction/species, so a rollout is self-contained (no ground-truth secondary counts injected). This is the "get a baseline out" track agreed with Jan & Tobias (2026-07-07). **Phase 2 (done):** the two-stage model jointly predicts `n_sec`, the energy simplex (`e_sec` falls out of it), and each secondary's energy/direction/species, so a rollout is self-contained (no ground-truth secondary counts injected).
**Physical-property conditioning (implemented):** `model.conditioning = "physical" | "embedding"` (see above) replaces the learned PDG/material embeddings with a small MLP over particle mass/charge and material Z_eff/A_eff/density/X0/λ_int, and Stage 2 predicts a secondary's mass/charge directly instead of a snapped species embedding. `"embedding"` stays available as the generalization-comparison baseline. `giant/materials.py`'s table is already filled with real values for every material the geometry produces. **Not yet done:** the actual held-out-material/species generalization comparison against the `"embedding"` baseline is unrun — the 34GB multi-material dataset at the repo root (6 materials, 237 PDG codes including nuclear/ion codes) is the natural dataset for that experiment. **Physical-property conditioning (implemented, default):** `conditioning.particle.type` / `conditioning.material.type` = `physical | embedding | onehot`. **Not yet done:** the actual held-out-material/species generalization comparison against the `"embedding"` baseline is unrun — the 34GB multi-material dataset (6 materials, 237 PDG codes including nuclear/ion codes) is the natural dataset for that experiment.
**Faster-eval architectures (implemented, validation in progress):** both tracks below target a ~10× native-Geant4 eval budget and are now wired into `giant train`/`giant/model/network.py`, but neither has a validated result yet — treat both as unproven until the corresponding analysis run says otherwise: **v0.3.0 — Stage-2 autoregressive redesign (implemented, released; on `master` since 2026-08-13):** motivated by the 2026-08-03 WGAN rollout benchmark, which failed specifically at the secondary-species level (zero photon secondaries, ~4M hallucinated `-14` muon antineutrinos). Stage 2 became autoregressive in descending-energy order with teacher forcing, and the particle-type representation went back to **categorical** (`particle_type.target = "onehot"`), reversing the 2026-07-17 continuous `(log-mass, charge)` target. The config break (`[conditioning]`/`[stage1_model]`/`[stage2_model]`/`[train]` replacing the flat `train.mode` + `[model]`) makes per-stage generators, stage-2-only training, and one-shot-vs-autoregressive comparison all expressible, and the `network.py` refactor into composable parts (encoder × trunk × objective) also makes routed WGAN work for the first time.
- **WGAN-GP** (`--mode wgan`, see Architecture above): implemented, **not yet tested** — no rollout-vs-reference analysis run against it yet.
- **MoE routing trunk** (`--router`, see Architecture above): implemented, **first rollout benchmark done (2026-07-22), result: needs retraining with a different router config, not abandoned.** A 10-expert `EnergyRouter` run (`n_experts=10`, `temperature=0.5`, `learn_centers=true`, **`lambda_balance=0.0`**, only 20 fine-tuning epochs resumed from a non-routed checkpoint) diverged badly from Geant4 on step granularity, secondary species, and shower shape, despite roughly matching bulk total deposited energy. The `router_gating` diagnostic plot points at the likely cause: the ten experts overlap heavily across ~5 decades of pre-step energy instead of partitioning it — even the top-energy expert only reaches ~6065% gate weight at the highest energies plotted — so eval-time top-1 (Voronoi) dispatch is choosing among near-ties rather than real specialists. Two contributors were identified: the missing load-balancing loss (`lambda_balance=0.0`), and `EnergyRouter`'s center init (`torch.linspace(-2, 2, n_experts)`) assuming a roughly uniform z-normalized energy distribution, which real energy spectra don't match. **Fixed (2026-07-27):** `EnergyRouter` now accepts an optional `centers_init` (backward compatible — omitting it keeps the old linspace), and `giant train` auto-populates it from real data quantiles via a reservoir sample collected during the existing normalizer-fitting pass in `giant/pipeline.py` (no extra file scan), for `--router-type energy` only. The routing *strategy* itself may still be sound, but the specific benchmarked config wasn't. **Next step before further evaluation: retrain with `lambda_balance > 0` and the new quantile-seeded centers (and consider more epochs / a from-scratch run rather than a short fine-tune), then re-check whether `router_gating` sharpens up.** Full writeup: `/home/lars/knowledge-base/experiments/giant-router-energy-rollout-validation.md`.
A sampling-calorimeter (multi-material) dataset is still a planned future direction, not yet built. See the knowledge base (`/home/lars/knowledge-base/meta/roadmap.md`). v0.2 configs and checkpoints are auto-migrated (`config.migrate_config`, `model._legacy._migrate_legacy_model_config`, both drawing on shared facts in `giant/_migration.py`). **v0.2 checkpoint-loading support has no expiry decided yet**: `/ceph` still holds pre-v0.3.0 checkpoints and analysis runs referencing them, so don't delete or substantially alter either migration function or `tests/legacy/network_v02_snapshot.py` (the frozen v0.2 snapshot they're tested against) without an explicit decision to do so first.
**v0.3.0 — Stage-2 autoregressive redesign (designed, not implemented; branch `v0.3.0-stage2-autoregressive`):** the 2026-08-03 WGAN rollout benchmark failed specifically at the secondary-species level (zero photon secondaries, ~4M hallucinated `-14` muon antineutrinos). The agreed response pivots Stage 2 to **autoregressive generation** in descending-energy order with teacher forcing, and switches the particle-type representation back to **categorical** (top N1 by training-set count + an "other" bucket), reversing the 2026-07-17 continuous `(log-mass, charge)` target. This requires a config break: `[conditioning]` / `[stage1_model]` / `[stage2_model]` / `[train]` blocks replace the single global `train.mode` + `[model]`, so per-stage generators (`stage1 = flow` + `stage2 = wgan`), stage-2-only training, and one-shot-vs-autoregressive comparison are all expressible. `network.py` is refactored from ten permutation classes into composable parts (encoder × trunk × objective), which also makes routed WGAN work for the first time. This config break is why v0.2-shaped configs/checkpoints need migrating at all (`config.migrate_config`, `model.network._migrate_legacy_model_config`, both drawing on shared facts in `giant/_migration.py`) — v0.2 checkpoint-loading support has **no expiry decided yet**: `/ceph` still holds pre-v0.3.0 checkpoints and analysis runs referencing them, so don't delete or substantially alter either migration function or `tests/legacy/network_v02_snapshot.py` (the frozen v0.2 snapshot they're tested against) without an explicit decision to do so first. **Faster-eval architectures — both implemented, neither validated.** Target is a ~10× native-Geant4 eval budget; no eval-latency number exists for any configuration yet, so that budget is unverified across the board.
- **WGAN-GP** (`--stage2-generator wgan`, now the stage-2 default): first rollout benchmark 2026-08-03 failed with secondary-species mode collapse — the failure v0.3.0 was designed to address. **No post-v0.3.0 benchmark has been run.** Writeup: `/home/lars/knowledge-base/experiments/giant-wgan-physical-rollout-validation.md`.
- **MoE routing trunk** (`--router`): first rollout benchmark 2026-07-22 diverged badly from Geant4 on step granularity, secondary species, and shower shape, despite roughly matching bulk total deposited energy. Cause identified as a bad config, not a bad idea: `lambda_balance=0.0` (no load-balancing loss) plus `EnergyRouter`'s `torch.linspace(-2, 2, n_experts)` center init assuming a roughly uniform z-normalized energy distribution — so the ten experts overlapped across ~5 decades of energy instead of partitioning it, and eval-time top-1 dispatch chose among near-ties rather than real specialists. Both prerequisites are fixed in code (quantile-seeded `centers_init` from `pipeline.py`, `lambda_balance` exposed). **Next step: retrain with `lambda_balance > 0` and quantile-seeded centers (consider a from-scratch run rather than a short fine-tune), then re-check whether `router_gating` sharpens up.** Writeup: `/home/lars/knowledge-base/experiments/giant-router-energy-rollout-validation.md`.
**Condor-submitted GPU training/rollout (in progress, `condor-gpu-train-rollout` branch, not yet merged):** moves `giant train`/`giant rollout` off the shared portal GPU dev machines (see Compute environment) onto remote-GPU HTCondor submission on TOpAS/NEMO2 (`giant/condor.py`). Partway between "needs major features" and feature-complete — not ready to merge yet. A sampling-calorimeter (multi-material) dataset track is still open and unblocked, not yet started. See the knowledge base (`/home/lars/knowledge-base/meta/roadmap.md`).
**Condor-submitted GPU training/rollout (in progress, `condor-gpu-train-rollout` branch, not yet merged):** moves `giant train`/`giant rollout` off the shared portal GPU dev machines (see Compute environment) onto remote-GPU HTCondor submission on TOpAS/NEMO2. Partway between "needs major features" and feature-complete — not ready to merge yet.
+48 -12
View File
@@ -10,6 +10,7 @@ A conditional generative model that replaces the Geant4 step function: given a p
uv sync --extra cpu # install deps (CPU torch; use --extra cuda for GPU) uv sync --extra cpu # install deps (CPU torch; use --extra cuda for GPU)
giant new-run --hidden-dim 512 --lr 3e-4 # scaffold config.toml + run dir giant new-run --hidden-dim 512 --lr 3e-4 # scaffold config.toml + run dir
giant model summary --config config.toml # parameter counts + which config keys actually bite
giant train path/to/steps.parquet # train (flow + wgan by default) giant train path/to/steps.parquet # train (flow + wgan by default)
giant predict path/to/steps.parquet --checkpoint checkpoints/.../best.pt giant predict path/to/steps.parquet --checkpoint checkpoints/.../best.pt
@@ -42,9 +43,9 @@ A **two-stage model**, checkpointed together. Either stage's outcome can be prod
Either way, secondary energies stick-break the `e_sec` budget handed down from Stage 1, so the full chain conserves energy. A secondary's particle identity is represented as `onehot` (categorical, top-N PDG codes + "other"), `physical` (continuous log-mass/charge), or `embedding` (nearest-neighbour lookup). Either way, secondary energies stick-break the `e_sec` budget handed down from Stage 1, so the full chain conserves energy. A secondary's particle identity is represented as `onehot` (categorical, top-N PDG codes + "other"), `physical` (continuous log-mass/charge), or `embedding` (nearest-neighbour lookup).
**Conditioning.** Pre-step position/energy/direction/layer, plus particle mass/charge and material Z_eff/A_eff/density/X0/λ_int, encoded the same three ways as particle identity above (`--conditioning`) — the `physical` representation generalizes to species/materials outside the training menu since it's computed rather than looked up. `n_sec`/`e_sec` are always model outputs, never conditioning inputs. **Conditioning.** Pre-step position/energy/direction/layer, plus particle mass/charge and material Z_eff/A_eff/density/X0/λ_int, encoded the same three ways as particle identity above. The particle and material axes are configured independently (`conditioning.particle.type` / `conditioning.material.type`; `--conditioning` sets both at once) and may mix — the `physical` representation generalizes to species/materials outside the training menu since it's computed rather than looked up. `n_sec`/`e_sec` are always model outputs, never conditioning inputs.
**MoE routing** (`--router`, either stage): a pluggable `Router` (`energy`/`pdg`/`process`/`composed` axes) top-1-dispatches each row to one of several small expert trunks at eval time, instead of running one monolithic trunk. **MoE routing** (`--router`): a pluggable `Router` (`energy`/`pdg`/`process`/`composed` axes) top-1-dispatches each row to one of several small expert trunks at eval time, instead of running one monolithic trunk. The CLI flags configure Stage 1's router; Stage 2 has its own `stage2_model.router` block, config-file only.
## Data ## Data
@@ -64,12 +65,24 @@ giant/
│ ├── data/ │ ├── data/
│ │ ├── loader.py # parquet → numpy arrays (incl. streaming/chunked reads) │ │ ├── loader.py # parquet → numpy arrays (incl. streaming/chunked reads)
│ │ ├── transforms.py # log transforms, local-frame rotation, energy simplex, secondary encode/decode │ │ ├── transforms.py # log transforms, local-frame rotation, energy simplex, secondary encode/decode
│ │ ── dataset.py # StepsDataset / StreamingStepsDataset (PyTorch) │ │ ── dataset.py # StepsDataset / StreamingStepsDataset (PyTorch)
│ │ └── setup_cache.py # sidecar cache for the pre-epoch setup scan (vocab/split/normalizers)
│ ├── model/ │ ├── model/
│ │ ├── network.py # ConditionEncoder, Stage1Model, Stage2OneShot/Stage2Autoregressive, Router/MoE, CriticModel │ │ ├── models.py # Stage1Model, Stage2OneShot, Stage2Autoregressive, CriticModel
│ │ ├── builders.py # build_models / build_critics — config dict → assembled stage models
│ │ ├── encoders.py # ConditionEncoder (physical / embedding / onehot, per axis)
│ │ ├── layers.py # ResBlock/AdaLNResBlock registry, SinusoidalEmbedding, MLP heads
│ │ ├── trunks.py # trunk registry (resmlp, none) + RoutedTrunk (MoE expert bodies)
│ │ ├── routers.py # Router registry: energy / pdg / process / composed / none
│ │ ├── history.py # stage-2 AR history encoders: markov / attention (KV-cached) / none
│ │ ├── objectives.py # flow / ddpm / wgan objective registry
│ │ ├── schedule.py # CosineSchedule (DDPM) and flow matching utilities │ │ ├── schedule.py # CosineSchedule (DDPM) and flow matching utilities
│ │ ── wgan.py # WGAN-GP gradient penalty / critic / generator losses │ │ ── wgan.py # WGAN-GP gradient penalty / critic / generator losses
│ │ ├── summary.py # build-only introspection behind `giant model summary`
│ │ ├── _legacy.py # v0.2 checkpoint model_config/state-dict migration
│ │ └── network.py # re-export shim over all of the above
│ ├── constants.py # output/conditioning dims, K_MAX, secondary slot layout, schema keys │ ├── constants.py # output/conditioning dims, K_MAX, secondary slot layout, schema keys
│ ├── cond_layout.py # single source of truth for the cond_cont/cond_cat column layout
│ ├── particles.py # PDG → (mass, charge) decode, incl. nuclear/ion codes; onehot/embedding secondary-identity decode │ ├── particles.py # PDG → (mass, charge) decode, incl. nuclear/ion codes; onehot/embedding secondary-identity decode
│ ├── materials.py # material name → (Z_eff, A_eff, density, X0, λ_int) │ ├── materials.py # material name → (Z_eff, A_eff, density, X0, λ_int)
│ ├── config.py # default hyperparameters, TOML config merging, device autodetect │ ├── config.py # default hyperparameters, TOML config merging, device autodetect
@@ -79,20 +92,28 @@ giant/
│ │ ├── trainers.py # StageSpec + flow/ddpm and WGAN-GP per-stage trainers │ │ ├── trainers.py # StageSpec + flow/ddpm and WGAN-GP per-stage trainers
│ │ ├── stage2_inputs.py# ground-truth stage-2 targets + autoregressive/teacher-forcing inputs │ │ ├── stage2_inputs.py# ground-truth stage-2 targets + autoregressive/teacher-forcing inputs
│ │ ├── metrics.py # MetricsCollector: metrics.csv columns, W&B logging, progress/summary │ │ ├── metrics.py # MetricsCollector: metrics.csv columns, W&B logging, progress/summary
│ │ ├── amp.py # bf16 autocast (`train.precision`)
│ │ ├── plots.py # training-progress plots (`giant analyze metrics`)
│ │ └── checkpoint.py # checkpoint assembly/restore (format unchanged since v0.2) │ │ └── checkpoint.py # checkpoint assembly/restore (format unchanged since v0.2)
│ ├── sample.py # DDPM / DDIM / flow matching / WGAN samplers + secondary sampling │ ├── sample.py # DDPM / DDIM / flow matching / WGAN samplers + secondary sampling
│ ├── checkpoint_io.py # checkpoint → ready-to-run models/normalizers (predict + rollout)
│ ├── geometry.py # GeometryOracle: position → (material, layer_id, escaped) for rollout │ ├── geometry.py # GeometryOracle: position → (material, layer_id, escaped) for rollout
│ ├── rollout.py # autoregressive shower rollout driver │ ├── rollout.py # autoregressive shower rollout driver
│ ├── validate.py # step-level marginal + KL-divergence validation │ ├── validate.py # step-level marginal + KL-divergence validation
│ ├── _migration.py # shared v0.2 → v0.3 facts used by both migration surfaces
│ ├── analysis/ # rollout-vs-reference analysis pipeline (see `giant analyze` below) │ ├── analysis/ # rollout-vs-reference analysis pipeline (see `giant analyze` below)
│ │ ├── sources.py # canonical LazyFrames + secondary view │ │ ├── sources.py # canonical LazyFrames + secondary view
│ │ ├── variables.py # per-step value expressions shared by range sizing and the catalog
│ │ ├── reduce.py # streaming reduction primitives (hist1d, per-event scalars, profiles, ...) │ │ ├── reduce.py # streaming reduction primitives (hist1d, per-event scalars, profiles, ...)
│ │ ├── grouping.py # fixed bin edges + energy/pdg/material group sets │ │ ├── grouping.py # fixed bin edges + energy/pdg/material group sets
│ │ ├── context.py # resolves grouping into `shared.json` once per run │ │ ├── context.py # resolves grouping into `shared.json` once per run
│ │ ├── catalog.py # declarative PlotSpec registry │ │ ├── reduced.py # Partial/Reduced — the compact JSON a compute job emits
│ │ ├── condor.py # prep / compute-one / submit-description plumbing │ │ ├── catalog.py # declarative PlotSpec registry (`giant analyze list`)
│ │ ├── router_gating.py / type_embedding_distance.py # checkpoint-bound diagnostics
│ │ ├── runtime_estimate.py # per-(plot, chunk) walltime estimates for submit
│ │ ├── condor.py # prep / compute-one / merge / submit-description plumbing
│ │ └── render.py # PDFs + HTML gallery (only module importing plotstyle/LaTeX) │ │ └── render.py # PDFs + HTML gallery (only module importing plotstyle/LaTeX)
│ └── cli.py # `giant train` / `new-run` / `predict` / `rollout` / `analyze` Typer app │ └── cli.py # `giant train` / `new-run` / `model summary` / `predict` / `rollout` / `analyze`
├── giant/tools/ # dataset/tooling logic, unified under the `dwarf` CLI (`dwarf --help`) ├── giant/tools/ # dataset/tooling logic, unified under the `dwarf` CLI (`dwarf --help`)
│ ├── dwarf.py # Typer app: convert, migrate, bump-gen, bump-schema, status, │ ├── dwarf.py # Typer app: convert, migrate, bump-gen, bump-schema, status,
│ │ # update-manifest, create-manifest, make-root, │ │ # update-manifest, create-manifest, make-root,
@@ -116,8 +137,13 @@ giant/
uv sync --extra cpu # CPU-only torch (use --extra cuda for CUDA 11.8 instead) uv sync --extra cpu # CPU-only torch (use --extra cuda for CUDA 11.8 instead)
uv sync --extra cpu --extra dev # add dev tools (pytest, ruff, ty) uv sync --extra cpu --extra dev # add dev tools (pytest, ruff, ty)
uv sync --extra cpu --extra geometry # add scikit-learn, for `dwarf build-geometry-oracle` / rollout uv sync --extra cpu --extra geometry # add scikit-learn, for `dwarf build-geometry-oracle` / rollout
uv sync --extra cpu --extra analysis # matplotlib/polars/plotstyle, for `giant analyze render`
uv sync --extra cpu --extra convert # uproot/awkward/polars, for `dwarf convert`
uv sync --extra cpu --extra wandb # W&B logging (`giant train --wandb`)
``` ```
The `dev` extra pulls in `convert`, `analysis`, `geometry` and `wandb` as well.
`cpu` and `cuda` are mutually exclusive — pick one to select the torch build (pinned to 2.3.x). Plain `uv sync` installs no torch at all. See `CLAUDE.md` for details. `cpu` and `cuda` are mutually exclusive — pick one to select the torch build (pinned to 2.3.x). Plain `uv sync` installs no torch at all. See `CLAUDE.md` for details.
## Training, prediction, rollout ## Training, prediction, rollout
@@ -137,11 +163,13 @@ Useful flags on `giant train`:
- `--stage2-decoder {autoregressive,one_shot}` — Stage 2 decoding strategy (see Architecture) - `--stage2-decoder {autoregressive,one_shot}` — Stage 2 decoding strategy (see Architecture)
- `--conditioning {physical,embedding,onehot}` — conditioning representation - `--conditioning {physical,embedding,onehot}` — conditioning representation
- `--router` / `--router-type` / `--n-experts` / `--router-axis` — MoE routing - `--router` / `--router-type` / `--n-experts` / `--router-axis` — MoE routing
- `--stage2-stage1-context {truth,sampled}` — feed Stage 2 the ground-truth or the model's own sampled Stage-1 outcome (annealable via `stage2_model.ctx_p_start`/`ctx_p_end`)
- `--precision {fp32,bf16}` — bf16 autocast in the training loop
- `--wandb` — log per-epoch metrics to Weights & Biases (needs `uv sync --extra wandb`); metric names are `<stage>/<split>/<metric>` plus an unprefixed run-level tail, all derived from `giant/training/trainers.py` `MetricSpec`s - `--wandb` — log per-epoch metrics to Weights & Biases (needs `uv sync --extra wandb`); metric names are `<stage>/<split>/<metric>` plus an unprefixed run-level tail, all derived from `giant/training/trainers.py` `MetricSpec`s
- `--no-cache-setup` / `--rebuild-setup-cache` — control the setup-stage sidecar cache (vocab maps, event split, normalizer stats); `dwarf warm-cache` precomputes it - `--no-cache-setup` / `--rebuild-setup-cache` — control the setup-stage sidecar cache (vocab maps, event split, normalizer stats); `dwarf warm-cache` precomputes it
- `--stage1-init-from`/`--stage2-init-from` (checkpoint `.pt`) + `--stage1-freeze`/`--stage2-freeze` — load a stage's weights from another checkpoint and never update them, so the other stage can be retrained alone against a fixed, known-good one while still producing a complete, rollout-capable checkpoint - `--stage1-init-from`/`--stage2-init-from` (checkpoint `.pt`) + `--stage1-freeze`/`--stage2-freeze` — load a stage's weights from another checkpoint and never update them, so the other stage can be retrained alone against a fixed, known-good one while still producing a complete, rollout-capable checkpoint
Config-file-only knobs (no CLI flag — use `--config config.toml`): `stage2_model.autoregressive.teacher_forcing`/`.history`, `stage2_model.particle_type.target`. v0.2 flat-schema configs and checkpoints load fine (auto-migrated). Config-file-only knobs (no CLI flag — use `--config config.toml`): `stage2_model.autoregressive.teacher_forcing`/`.history`, `stage2_model.particle_type.target`/`.class_weighting`, `stage2_model.n_sec.mode`/`.owner`, `conditioning.share_stages`, `stage*_model.trunk.*` and the finer `router` knobs (`lambda_balance`, `gumbel`, `learn_width`, …). `configs/` holds kept reference configs. v0.2 flat-schema configs and checkpoints load fine (auto-migrated).
`giant rollout` seeds showers from each event's highest-energy entry step, then autoregressively steps the model to completion, pushing secondaries as new tracks and looking up `material`/`layer_id` from the geometry oracle each step. Tracks terminate on energy cutoff, max steps, detector escape, or natural end; energy is deposited locally on every stop except escape, so showers conserve energy by construction. `giant rollout` seeds showers from each event's highest-energy entry step, then autoregressively steps the model to completion, pushing secondaries as new tracks and looking up `material`/`layer_id` from the geometry oracle each step. Tracks terminate on energy cutoff, max steps, detector escape, or natural end; energy is deposited locally on every stop except escape, so showers conserve energy by construction.
@@ -151,11 +179,19 @@ Config-file-only knobs (no CLI flag — use `--config config.toml`): `stage2_mod
- `giant analyze` — deeper rollout-vs-reference diagnostics (marginals by energy/pdg/material, per-event totals, shower profiles, species share, leakage, secondaries): - `giant analyze` — deeper rollout-vs-reference diagnostics (marginals by energy/pdg/material, per-event totals, shower profiles, species share, leakage, secondaries):
```bash ```bash
giant analyze submit rollout.yaml --accounting-group cms # prep + one HTCondor job per plot (compute only) giant analyze submit rollout.yaml --accounting-group cms # prep + one HTCondor job per plot × chunk (compute only)
giant analyze render <run_dir> --gallery # local: styled PDFs + HTML gallery (needs LaTeX) giant analyze submit a.yaml b.yaml --accounting-group cms --label flow --label wgan # N rollouts vs one shared reference
giant analyze render <run_dir> --gallery # local: merge chunks, then styled PDFs + HTML gallery (needs LaTeX)
giant analyze list # every catalog plot id
giant analyze prep rollout.yaml --chunks 8 # just the run directory, no submission
giant analyze compute-one --id marginal_edep --run-dir <run_dir> --chunk 0 # what a condor job runs
giant analyze merge-one --id marginal_edep --run-dir <run_dir> # merge one plot's chunks (debugging)
``` ```
`<run_dir>` is derived next to the rollout parquet (`analyze prep`/`submit` print it). Compute jobs are polars/numpy only; only `render` needs LaTeX, so it always runs locally. `<run_dir>` defaults to `<cwd>/analysis_runs/analysis_<id>` (`--run-dir` overrides it; `prep`/`submit` print it). Multiple rollout YAMLs must all name the same reference (`dataset`) file; each renders as its own colored series against one reference line/panel. Compute jobs are polars/numpy only; only `render` needs LaTeX, so it always runs locally.
Separately, `giant analyze metrics <train_run_dir>` renders training-progress plots (loss/lr/accuracy/grad-norm/router/wgan/throughput) straight from a training run's `metrics.csv`.
## Development ## Development
+11 -5
View File
@@ -1,9 +1,10 @@
"""Rollout-vs-reference analysis: streaming compute + plotstyle rendering. """Rollout-vs-reference analysis: streaming compute + plotstyle rendering.
Compares one autoregressive ``giant rollout`` against a held-out miniCaloSim Compares one or more autoregressive ``giant rollout`` runs against a single
reference file, producing publication-styled comparison plots generated in held-out miniCaloSim reference file shared by all of them, producing
parallel on HTCondor (one job per plot x data chunk, compute/merge/render publication-styled comparison plots (one colored series per rollout, one
split). reference line) generated in parallel on HTCondor (one job per plot x data
chunk, compute/merge/render split).
Only ``render`` (and the ``render`` CLI path) imports plotstyle/LaTeX; everything Only ``render`` (and the ``render`` CLI path) imports plotstyle/LaTeX; everything
re-exported here is plotstyle-free so it runs on a compute worker. Import re-exported here is plotstyle-free so it runs on a compute worker. Import
@@ -12,12 +13,14 @@ re-exported here is plotstyle-free so it runs on a compute worker. Import
from giant.analysis.catalog import build_catalog, catalog_ids, get_spec from giant.analysis.catalog import build_catalog, catalog_ids, get_spec
from giant.analysis.condor import ( from giant.analysis.condor import (
LoadedRollout,
RunMeta, RunMeta,
SubmitConfig, SubmitConfig,
compute_one, compute_one,
compute_reduced, compute_reduced,
derive_run_dir, derive_run_dir,
load_rollout_yaml, load_rollout_yaml,
load_rollout_yamls,
merge_all, merge_all,
merge_one, merge_one,
prep, prep,
@@ -26,18 +29,20 @@ from giant.analysis.condor import (
from giant.analysis.context import Context, build_context from giant.analysis.context import Context, build_context
from giant.analysis.reduced import Partial, Reduced from giant.analysis.reduced import Partial, Reduced
from giant.analysis.runtime_estimate import RUNTIME_SAFETY_MARGIN, estimate_runtime_s from giant.analysis.runtime_estimate import RUNTIME_SAFETY_MARGIN, estimate_runtime_s
from giant.analysis.sources import Side from giant.analysis.sources import RolloutSpec, Side
__all__ = [ __all__ = [
"build_catalog", "build_catalog",
"catalog_ids", "catalog_ids",
"get_spec", "get_spec",
"LoadedRollout",
"RunMeta", "RunMeta",
"SubmitConfig", "SubmitConfig",
"compute_one", "compute_one",
"compute_reduced", "compute_reduced",
"derive_run_dir", "derive_run_dir",
"load_rollout_yaml", "load_rollout_yaml",
"load_rollout_yamls",
"merge_all", "merge_all",
"merge_one", "merge_one",
"prep", "prep",
@@ -46,6 +51,7 @@ __all__ = [
"build_context", "build_context",
"Partial", "Partial",
"Reduced", "Reduced",
"RolloutSpec",
"Side", "Side",
"RUNTIME_SAFETY_MARGIN", "RUNTIME_SAFETY_MARGIN",
"estimate_runtime_s", "estimate_runtime_s",
+547 -161
View File
File diff suppressed because it is too large Load Diff
+132 -45
View File
@@ -1,4 +1,4 @@
"""HTCondor orchestration driven by a ``giant rollout`` YAML sidecar. """HTCondor orchestration driven by one or more ``giant rollout`` YAML sidecars.
A rollout writes a YAML sidecar (``giant/cli.py:_write_prediction_ref`` + A rollout writes a YAML sidecar (``giant/cli.py:_write_prediction_ref`` +
rollout extras) that already names both files we need and carries the run's rollout extras) that already names both files we need and carries the run's
@@ -10,8 +10,11 @@ provenance:
* ``checkpoint``, ``geometry_oracle``, ``energy_cutoff``, ``steps``, ... * ``checkpoint``, ``geometry_oracle``, ``energy_cutoff``, ``steps``, ...
metadata that flows straight into every plot's gallery ``metadata.yaml``. metadata that flows straight into every plot's gallery ``metadata.yaml``.
So the analysis takes that one YAML as input, derives its own **run directory** The analysis takes N such YAMLs one series per rollout, all required to
next to the rollout parquet, and lays everything out under it: share the same ``dataset`` (the premise is "N candidates vs one ground
truth") — resolves each one's series name (``load_rollout_yamls``), derives
its own **run directory** next to the first rollout's parquet, and lays
everything out under it:
<run_dir>/shared.json fixed bin edges / group sets (prep) <run_dir>/shared.json fixed bin edges / group sets (prep)
<run_dir>/run_meta.json resolved rollout/reference paths + plot metadata <run_dir>/run_meta.json resolved rollout/reference paths + plot metadata
@@ -44,6 +47,7 @@ from __future__ import annotations
import json import json
import shutil import shutil
import sys import sys
from collections.abc import Sequence
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
@@ -54,7 +58,7 @@ from giant.analysis.catalog import Bundle, catalog_ids, get_spec
from giant.analysis.context import Context, build_context from giant.analysis.context import Context, build_context
from giant.analysis.reduced import Partial from giant.analysis.reduced import Partial
from giant.analysis.runtime_estimate import estimate_runtime_s from giant.analysis.runtime_estimate import estimate_runtime_s
from giant.analysis.sources import Side, open_side from giant.analysis.sources import RolloutSpec, Side, open_side
# Keys copied verbatim from a rollout YAML into each plot's gallery metadata. # Keys copied verbatim from a rollout YAML into each plot's gallery metadata.
_PLOT_META_KEYS = ( _PLOT_META_KEYS = (
@@ -111,8 +115,60 @@ def load_rollout_yaml(path: str | Path) -> dict:
return d return d
@dataclass
class LoadedRollout:
"""One rollout YAML plus its resolved series ``name`` (see ``load_rollout_yamls``)."""
name: str
yaml: dict
def load_rollout_yamls(
paths: Sequence[str | Path], labels: Sequence[str] | None = None
) -> tuple[list[LoadedRollout], str]:
"""Load every rollout YAML, resolve each one's series name, and verify they
all share one reference (``dataset``) file the premise is "N candidates
vs one ground truth", not N independent comparisons.
Names: an explicit ``labels[i]`` if given (``labels`` must be empty or
exactly ``len(paths)`` long); otherwise the YAML's stem for N>1, or
``"rollout"`` for the single-YAML case matching today's one-series
legend/payload key, so a single-rollout run renders identically to
before this feature existed. Raises ``ValueError`` if two rollouts
resolve to the same name, or if the YAMLs don't all name the same
``dataset``.
"""
if labels and len(labels) != len(paths):
raise ValueError(f"--label given {len(labels)} time(s) but {len(paths)} rollout YAML(s) were passed")
yamls = [load_rollout_yaml(p) for p in paths]
if labels:
names = list(labels)
elif len(paths) == 1:
names = ["rollout"]
else:
names = [Path(p).stem for p in paths]
if len(set(names)) != len(names):
dupes = sorted({n for n in names if names.count(n) > 1})
raise ValueError(f"rollout series names collide: {dupes} — pass --label to disambiguate")
references = {str(y["dataset"]) for y in yamls}
if len(references) > 1:
detail = "\n".join(f" {p}: dataset={y['dataset']!r}" for p, y in zip(paths, yamls))
raise ValueError(
"all rollout YAMLs must be seeded from the same reference (dataset) "
f"file — got {len(references)} distinct ones:\n{detail}"
)
return [LoadedRollout(name=n, yaml=y) for n, y in zip(names, yamls)], yamls[0]["dataset"]
def _run_tag(y: dict) -> str:
rollout = Path(y["output"])
return str(y.get("prediction_id") or rollout.stem)[:8]
def derive_run_dir( def derive_run_dir(
rollout_yaml: dict, rollout_yamls: list[dict],
run_dir: str | Path | None = None, run_dir: str | Path | None = None,
default_base: str | Path | None = None, default_base: str | Path | None = None,
) -> Path: ) -> Path:
@@ -122,14 +178,24 @@ def derive_run_dir(
``default_base / analysis_<tag>`` if ``default_base`` is given (the CLI ``default_base / analysis_<tag>`` if ``default_base`` is given (the CLI
passes the repo's gitignored ``analysis_runs/``, so run directories don't passes the repo's gitignored ``analysis_runs/``, so run directories don't
pile up on ``/ceph`` next to the rollout parquet). Falls back to next to pile up on ``/ceph`` next to the rollout parquet). Falls back to next to
the rollout parquet the original convention for callers that don't the *first* rollout's parquet — the original convention — for callers
care where the run directory lives. that don't care where the run directory lives.
``tag`` is a single rollout's ``prediction_id``/output stem (matching
today's single-rollout convention exactly) when there's only one; for
N>1 it joins up to three tags with ``-``, then ``-plus<K>`` for any
beyond that, so a many-rollout run still gets a short, stable directory
name.
""" """
if run_dir is not None: if run_dir is not None:
return Path(run_dir) return Path(run_dir)
rollout = Path(rollout_yaml["output"]) tags = [_run_tag(y) for y in rollout_yamls]
tag = str(rollout_yaml.get("prediction_id") or rollout.stem)[:8] if len(tags) == 1:
base = Path(default_base) if default_base is not None else rollout.parent tag = tags[0]
else:
shown, rest = tags[:3], tags[3:]
tag = "-".join(shown) + (f"-plus{len(rest)}" if rest else "")
base = Path(default_base) if default_base is not None else Path(rollout_yamls[0]["output"]).parent
return base / f"analysis_{tag}" return base / f"analysis_{tag}"
@@ -139,17 +205,21 @@ def _plot_meta(rollout_yaml: dict) -> dict:
@dataclass @dataclass
class RunMeta: class RunMeta:
"""Resolved paths + plot metadata for one analysis run (``run_meta.json``).""" """Resolved paths + plot metadata for one analysis run (``run_meta.json``).
rollout: str ``rollouts`` is ``[{"name", "path", "plot_meta"}, ...]``, insertion order
= the order rollouts were given on the CLI (and so the order every
``Reduced.payload["series"]`` dict is built in see ``catalog.py``).
"""
rollouts: list[dict]
reference: str reference: str
run_dir: str run_dir: str
title: str title: str
plot_meta: dict
n_chunks: int = 1 n_chunks: int = 1
# rollout+reference row count of each event_id-disjoint chunk, and the # combined rollout+reference row count of each event_id-disjoint chunk,
# dataset total — inputs to `runtime_estimate.estimate_runtime_s`. Empty/0 # and the dataset total — inputs to `runtime_estimate.estimate_runtime_s`.
# on run directories written before this field existed. # Empty/0 on run directories written before this field existed.
rows_per_chunk: list[int] = field(default_factory=list) rows_per_chunk: list[int] = field(default_factory=list)
total_rows: int = 0 total_rows: int = 0
@@ -161,8 +231,8 @@ class RunMeta:
return cls(**json.loads(Path(path).read_text())) return cls(**json.loads(Path(path).read_text()))
def _rows_per_chunk(rollout: str | Path, reference: str | Path, n_chunks: int) -> list[int]: def _rows_per_chunk(rollouts: list[str | Path], reference: str | Path, n_chunks: int) -> list[int]:
"""Rollout+reference row count of each ``event_id % n_chunks`` chunk. """Combined rollout+reference row count of each ``event_id % n_chunks`` chunk.
One cheap streaming ``group_by`` per side (just the ``event_id`` column) One cheap streaming ``group_by`` per side (just the ``event_id`` column)
the sizing input every job's estimated walltime the sizing input every job's estimated walltime
@@ -178,7 +248,8 @@ def _rows_per_chunk(rollout: str | Path, reference: str | Path, n_chunks: int) -
) )
out = [0] * n_chunks out = [0] * n_chunks
for lf in (open_side(rollout, Side.rollout), open_side(reference, Side.reference)): sides = [open_side(reference, Side.reference)] + [open_side(r, Side.rollout) for r in rollouts]
for lf in sides:
df = counts(lf) df = counts(lf)
for c, n in zip(df["_c"].to_list(), df["n"].to_list()): for c, n in zip(df["_c"].to_list(), df["n"].to_list()):
out[c] += n out[c] += n
@@ -186,20 +257,22 @@ def _rows_per_chunk(rollout: str | Path, reference: str | Path, n_chunks: int) -
def prep( def prep(
rollout_yaml: str | Path, rollout_yamls: Sequence[str | Path],
run_dir: str | Path | None = None, run_dir: str | Path | None = None,
n_chunks: int = 1, n_chunks: int = 1,
default_base: str | Path | None = None, default_base: str | Path | None = None,
labels: Sequence[str] | None = None,
**ctx_kwargs, **ctx_kwargs,
) -> Path: ) -> Path:
"""Read the rollout YAML, build the shared context, and lay out the run dir. """Read the rollout YAML(s), build the shared context, and lay out the run dir.
Writes ``shared.json`` + ``run_meta.json`` and returns the run directory. Writes ``shared.json`` + ``run_meta.json`` and returns the run directory.
``n_chunks`` is the run-level chunk count every ``compute-one``/``merge-one`` ``n_chunks`` is the run-level chunk count every ``compute-one``/``merge-one``
job reads back out of ``run_meta.json`` (via ``RunMeta.n_chunks``), so it is job reads back out of ``run_meta.json`` (via ``RunMeta.n_chunks``), so it is
resolved once here rather than re-passed (and risking disagreement) at every resolved once here rather than re-passed (and risking disagreement) at every
later step. See ``derive_run_dir`` for how ``run_dir``/``default_base`` later step. See ``derive_run_dir`` for how ``run_dir``/``default_base``
resolve the actual directory. resolve the actual directory, and ``load_rollout_yamls`` for how
``labels``/YAML stems resolve each rollout's series name.
Clears any existing ``reduced_partial/``/``reduced/`` from a prior prep of Clears any existing ``reduced_partial/``/``reduced/`` from a prior prep of
this same ``run_dir``: partial files carry no record of what context this same ``run_dir``: partial files carry no record of what context
@@ -208,8 +281,8 @@ def prep(
rollout/reference files changed) would otherwise let ``merge_one`` silently rollout/reference files changed) would otherwise let ``merge_one`` silently
merge stale partials against the new ``shared.json``. merge stale partials against the new ``shared.json``.
""" """
y = load_rollout_yaml(rollout_yaml) loaded, reference = load_rollout_yamls(list(rollout_yamls), labels)
run_path = derive_run_dir(y, run_dir, default_base=default_base) run_path = derive_run_dir([lr.yaml for lr in loaded], run_dir, default_base=default_base)
run_path.mkdir(parents=True, exist_ok=True) run_path.mkdir(parents=True, exist_ok=True)
for stale in ("reduced_partial", "reduced"): for stale in ("reduced_partial", "reduced"):
@@ -217,19 +290,22 @@ def prep(
if stale_dir.exists(): if stale_dir.exists():
shutil.rmtree(stale_dir) shutil.rmtree(stale_dir)
rollout, reference = y["output"], y["dataset"] rollout_specs = [RolloutSpec(name=lr.name, source=lr.yaml["output"]) for lr in loaded]
ctx = build_context(rollout, reference, **ctx_kwargs) ctx = build_context(rollout_specs, reference, **ctx_kwargs)
ctx.save(run_path / "shared.json") ctx.save(run_path / "shared.json")
rows_per_chunk = _rows_per_chunk(rollout, reference, n_chunks) rows_per_chunk = _rows_per_chunk([lr.yaml["output"] for lr in loaded], reference, n_chunks)
rollouts_meta = [
{"name": lr.name, "path": str(lr.yaml["output"]), "plot_meta": _plot_meta(lr.yaml)} for lr in loaded
]
ckpts = ", ".join(Path(lr.yaml.get("checkpoint", "")).name or "rollout" for lr in loaded)
ckpt = Path(y.get("checkpoint", "")).name or "rollout"
RunMeta( RunMeta(
rollout=str(rollout), rollouts=rollouts_meta,
reference=str(reference), reference=str(reference),
run_dir=str(run_path), run_dir=str(run_path),
title=f"GIANT rollout analysis — {ckpt}", title=f"GIANT rollout analysis — {ckpts}",
plot_meta=_plot_meta(y),
n_chunks=n_chunks, n_chunks=n_chunks,
rows_per_chunk=rows_per_chunk, rows_per_chunk=rows_per_chunk,
total_rows=sum(rows_per_chunk), total_rows=sum(rows_per_chunk),
@@ -244,17 +320,19 @@ def prep(
def compute_reduced( def compute_reduced(
spec_id: str, spec_id: str,
rollout: str | Path, rollouts: list[dict],
reference: str | Path, reference: str | Path,
shared: str | Path, shared: str | Path,
out: str | Path, out: str | Path,
checkpoint: str | None = None,
chunk_index: int = 0, chunk_index: int = 0,
n_chunks: int = 1, n_chunks: int = 1,
type_embedding_l1_dist: dict | None = None,
) -> Path: ) -> Path:
"""Core: run one (plot, chunk)'s partial reduction against explicit paths. """Core: run one (plot, chunk)'s partial reduction against explicit paths.
``rollouts``: ``[{"name", "path", "checkpoint"?, "type_embedding_l1_dist"?},
...]``, one per rollout series (insertion order preserved through to every
plot's ``Reduced.payload["series"]``).
Writes a ``Partial`` JSON the raw, not-yet-merged output of Writes a ``Partial`` JSON the raw, not-yet-merged output of
``PlotSpec.compute_partial`` never a finished ``Reduced``; ``merge_one`` ``PlotSpec.compute_partial`` never a finished ``Reduced``; ``merge_one``
is what combines every chunk's ``Partial`` for a plot into the final is what combines every chunk's ``Partial`` for a plot into the final
@@ -268,14 +346,16 @@ def compute_reduced(
raise ValueError( raise ValueError(
f"{spec_id}: chunk_index={chunk_index} out of range for n_chunks={effective_n} (chunkable={spec.chunkable})" f"{spec_id}: chunk_index={chunk_index} out of range for n_chunks={effective_n} (chunkable={spec.chunkable})"
) )
bundle = Bundle.open( rollout_specs = [
rollout, RolloutSpec(
reference, name=r["name"],
ctx, source=r["path"],
checkpoint=checkpoint, checkpoint=r.get("checkpoint"),
chunk=(chunk_index, effective_n), type_embedding_l1_dist=r.get("type_embedding_l1_dist"),
type_embedding_l1_dist=type_embedding_l1_dist, )
) for r in rollouts
]
bundle = Bundle.open(rollout_specs, reference, ctx, chunk=(chunk_index, effective_n))
partial = Partial( partial = Partial(
id=spec_id, id=spec_id,
family=spec.family, family=spec.family,
@@ -291,16 +371,23 @@ def compute_one(spec_id: str, run_dir: str | Path, chunk_index: int = 0) -> Path
"""Run one (plot, chunk)'s partial reduction from a prepped run directory.""" """Run one (plot, chunk)'s partial reduction from a prepped run directory."""
run_path = Path(run_dir) run_path = Path(run_dir)
meta = RunMeta.load(run_path / "run_meta.json") meta = RunMeta.load(run_path / "run_meta.json")
rollouts = [
{
"name": ro["name"],
"path": ro["path"],
"checkpoint": ro["plot_meta"].get("checkpoint"),
"type_embedding_l1_dist": ro["plot_meta"].get("type_embedding_l1_dist"),
}
for ro in meta.rollouts
]
return compute_reduced( return compute_reduced(
spec_id, spec_id,
meta.rollout, rollouts,
meta.reference, meta.reference,
run_path / "shared.json", run_path / "shared.json",
run_path / "reduced_partial" / f"{spec_id}__{chunk_index}.json", run_path / "reduced_partial" / f"{spec_id}__{chunk_index}.json",
checkpoint=meta.plot_meta.get("checkpoint"),
chunk_index=chunk_index, chunk_index=chunk_index,
n_chunks=meta.n_chunks, n_chunks=meta.n_chunks,
type_embedding_l1_dist=meta.plot_meta.get("type_embedding_l1_dist"),
) )
+36 -25
View File
@@ -26,7 +26,7 @@ from giant.analysis.reduce import (
entry_axis, entry_axis,
transverse_expr, transverse_expr,
) )
from giant.analysis.sources import Side, open_side, physical_steps, secondaries from giant.analysis.sources import RolloutSpec, Side, open_side, physical_steps, secondaries
from giant.analysis.variables import RANGED_VARS from giant.analysis.variables import RANGED_VARS
@@ -74,9 +74,9 @@ def _row_subsample(lf: pl.LazyFrame, sample_rows: int, seed: int) -> pl.LazyFram
return lf.filter((pl.col("pre_E").hash(seed=seed) % 2**32) < threshold) return lf.filter((pl.col("pre_E").hash(seed=seed) % 2**32) < threshold)
def _combined_quantiles(r_vals: np.ndarray, t_vals: np.ndarray, lo_q: float, hi_q: float) -> tuple[float, float]: def _combined_quantiles(vals: list[np.ndarray], lo_q: float, hi_q: float) -> tuple[float, float]:
"""Robust (lo_q, hi_q) range over the union of two value samples.""" """Robust (lo_q, hi_q) range over the union of several value samples."""
both = np.concatenate([r_vals, t_vals]) both = np.concatenate(vals)
lo, hi = float(np.quantile(both, lo_q)), float(np.quantile(both, hi_q)) lo, hi = float(np.quantile(both, lo_q)), float(np.quantile(both, hi_q))
if not (hi - lo > 1e-6 * max(abs(hi), 1.0)): if not (hi - lo > 1e-6 * max(abs(hi), 1.0)):
lo, hi = lo - 0.5, hi + 0.5 lo, hi = lo - 0.5, hi + 0.5
@@ -84,7 +84,7 @@ def _combined_quantiles(r_vals: np.ndarray, t_vals: np.ndarray, lo_q: float, hi_
def build_context( def build_context(
rollout: str | Path | pl.LazyFrame, rollouts: list[RolloutSpec],
reference: str | Path | pl.LazyFrame, reference: str | Path | pl.LazyFrame,
*, *,
n_energy_bins: int = 4, n_energy_bins: int = 4,
@@ -94,41 +94,51 @@ def build_context(
sample_rows: int = 1_000_000, sample_rows: int = 1_000_000,
seed: int = 0, seed: int = 0,
) -> Context: ) -> Context:
"""Resolve the shared context from the two files (the ``prep`` step).""" """Resolve the shared context from the reference + every rollout (the ``prep`` step).
r_all = open_side(rollout, Side.rollout)
Every range/quantile below is the union of the reference and *all*
rollouts, so a single set of fixed bin edges/group sets is valid for
every series a compute job streams over.
"""
t_all = open_side(reference, Side.reference) t_all = open_side(reference, Side.reference)
r_lf = physical_steps(r_all, Side.rollout)
t_lf = physical_steps(t_all, Side.reference) t_lf = physical_steps(t_all, Side.reference)
r_lfs = {rs.name: physical_steps(open_side(rs.source, Side.rollout), Side.rollout) for rs in rollouts}
# Ranged marginal variables: robust ranges over a shared row subsample. # Ranged marginal variables: robust ranges over a shared row subsample.
exprs = [e.alias(n) for n, (_, e) in RANGED_VARS.items()] exprs = [e.alias(n) for n, (_, e) in RANGED_VARS.items()]
r_s = _row_subsample(r_lf, sample_rows, seed).select(exprs).collect(engine="streaming")
t_s = _row_subsample(t_lf, sample_rows, seed).select(exprs).collect(engine="streaming") t_s = _row_subsample(t_lf, sample_rows, seed).select(exprs).collect(engine="streaming")
r_s = {
name: _row_subsample(lf, sample_rows, seed).select(exprs).collect(engine="streaming")
for name, lf in r_lfs.items()
}
var_ranges = { var_ranges = {
name: _combined_quantiles(r_s[name].to_numpy(), t_s[name].to_numpy(), _LO_Q, _HI_Q) for name in RANGED_VARS name: _combined_quantiles([t_s[name].to_numpy(), *(df[name].to_numpy() for df in r_s.values())], _LO_Q, _HI_Q)
for name in RANGED_VARS
} }
# Energy-bin edges from exact per-event incident energies (cheap group_by). # Energy-bin edges from exact per-event incident energies (cheap group_by).
def _incident(lf: pl.LazyFrame) -> np.ndarray: def _incident(lf: pl.LazyFrame) -> np.ndarray:
return lf.group_by("event_id").agg(pl.col("pre_E").max()).collect(engine="streaming")["pre_E"].to_numpy() return lf.group_by("event_id").agg(pl.col("pre_E").max()).collect(engine="streaming")["pre_E"].to_numpy()
r_inc, t_inc = _incident(r_lf), _incident(t_lf) t_inc = _incident(t_lf)
energy_edges = energy_bin_edges(np.concatenate([r_inc, t_inc]), n_energy_bins) r_inc = {name: _incident(lf) for name, lf in r_lfs.items()}
energy_edges = energy_bin_edges(np.concatenate([t_inc, *r_inc.values()]), n_energy_bins)
# Top PDG species and material list (cheap single-column group_bys). # Top PDG species and material list (cheap single-column group_bys).
def _counts(lf: pl.LazyFrame, col: str) -> pl.DataFrame: def _counts(lf: pl.LazyFrame, col: str) -> pl.DataFrame:
return lf.group_by(col).agg(pl.len().alias("n")).collect(engine="streaming") return lf.group_by(col).agg(pl.len().alias("n")).collect(engine="streaming")
pdg_counts = ( pdg_counts = (
pl.concat([_counts(r_lf, "pdg"), _counts(t_lf, "pdg")]) pl.concat([_counts(t_lf, "pdg"), *(_counts(lf, "pdg") for lf in r_lfs.values())])
.group_by("pdg") .group_by("pdg")
.agg(pl.col("n").sum()) .agg(pl.col("n").sum())
.sort("n", descending=True) .sort("n", descending=True)
) )
top_pdgs = [int(x) for x in pdg_counts["pdg"].to_list()[:top_k_pdg]] top_pdgs = [int(x) for x in pdg_counts["pdg"].to_list()[:top_k_pdg]]
materials = sorted( material_set: set[str] = set(_counts(t_lf, "material")["material"].to_list())
set(_counts(r_lf, "material")["material"].to_list()) | set(_counts(t_lf, "material")["material"].to_list()) for lf in r_lfs.values():
) material_set |= set(_counts(lf, "material")["material"].to_list())
materials = sorted(material_set)
# Shower depth / transverse ranges from a subsampled proxy. # Shower depth / transverse ranges from a subsampled proxy.
def _proxy(lf: pl.LazyFrame) -> tuple[np.ndarray, np.ndarray]: def _proxy(lf: pl.LazyFrame) -> tuple[np.ndarray, np.ndarray]:
@@ -140,19 +150,20 @@ def build_context(
) )
return sub["d"].to_numpy(), sub["t"].to_numpy() return sub["d"].to_numpy(), sub["t"].to_numpy()
r_d, r_t = _proxy(r_lf)
t_d, t_t = _proxy(t_lf) t_d, t_t = _proxy(t_lf)
d_lo, d_hi = _combined_quantiles(r_d, t_d, _LO_Q, _HI_Q) r_proxy = {name: _proxy(lf) for name, lf in r_lfs.items()}
d_lo, d_hi = _combined_quantiles([t_d, *(p[0] for p in r_proxy.values())], _LO_Q, _HI_Q)
depth_edges = np.linspace(d_lo, d_hi, n_marginal_bins + 1) depth_edges = np.linspace(d_lo, d_hi, n_marginal_bins + 1)
t_hi = max(float(np.quantile(np.concatenate([r_t, t_t]), _HI_Q)), 1e-6) t_hi = max(float(np.quantile(np.concatenate([t_t, *(p[1] for p in r_proxy.values())]), _HI_Q)), 1e-6)
transverse_edges = np.linspace(0.0, t_hi, n_marginal_bins + 1) transverse_edges = np.linspace(0.0, t_hi, n_marginal_bins + 1)
# Secondary energy range. # Secondary energy range.
r_se = secondaries(r_lf, Side.rollout).select("energy") t_se = _row_sample_col(secondaries(t_all, Side.reference).select("energy"), sample_rows, seed)
t_se = secondaries(t_all, Side.reference).select("energy") r_se = {
r_se = _row_sample_col(r_se, sample_rows, seed) name: _row_sample_col(secondaries(lf, Side.rollout).select("energy"), sample_rows, seed)
t_se = _row_sample_col(t_se, sample_rows, seed) for name, lf in r_lfs.items()
sec_energy_range = _combined_quantiles(r_se, t_se, _LO_Q, _HI_Q) }
sec_energy_range = _combined_quantiles([t_se, *r_se.values()], _LO_Q, _HI_Q)
return Context( return Context(
n_marginal_bins=n_marginal_bins, n_marginal_bins=n_marginal_bins,
@@ -165,8 +176,8 @@ def build_context(
sec_energy_range=sec_energy_range, sec_energy_range=sec_energy_range,
n_sec_bins=n_sec_bins, n_sec_bins=n_sec_bins,
n_events={ n_events={
"rollout": len(r_inc),
"reference": len(t_inc), "reference": len(t_inc),
**{name: len(arr) for name, arr in r_inc.items()},
}, },
) )
+17 -8
View File
@@ -11,15 +11,24 @@ import json
from dataclasses import asdict, dataclass, field from dataclasses import asdict, dataclass, field
from pathlib import Path from pathlib import Path
# Reduced.kind values: # Reduced.kind values (payload keys a rollout series by name under
# "overlay_hist" rollout vs reference density histogram over shared edges # payload["series"], with the reference — where one exists — kept as one
# distinguished payload["reference"] entry; see catalog.py's module
# docstring for the full per-kind payload shape):
# "overlay_hist" N-rollout-series vs reference density histogram over shared edges
# "grouped_hist" one panel per group (energy/pdg/material), each an overlay # "grouped_hist" one panel per group (energy/pdg/material), each an overlay
# "profile" edep-weighted mean +/- event-RMS vs depth/radius, two series # "profile" edep-weighted mean +/- event-RMS vs depth/radius, N series + reference
# "bar" per-category rollout vs reference bars (share / counts) # "bar" per-category N-rollout-series vs reference bars (share / counts)
# "single_hist" one series only (e.g. rollout leakage; reference has none) # "single_hist" rollout-only series (e.g. leakage; reference has none)
# "router_gating" stacked mean MoE gate weight vs energy, rollout + reference # "router_gating" stacked mean MoE gate weight vs energy, one rollout+reference
# "router_share" stacked bar of MoE top-1 dispatch share by category # panel-pair per rollout with an enabled MoE router
# "unavailable" plot not applicable to this run (e.g. non-MoE checkpoint) # "router_share" stacked bar of MoE top-1 dispatch share by category, one
# panel per rollout with an enabled MoE router
# "router_specialization" max gate weight vs energy (one scalar trend line
# summarizing "router_gating"), per rollout with an enabled router
# "heatmap" row x col matrix + colorbar, one panel per rollout (a
# distance scorecard)
# "unavailable" plot not applicable to this run (e.g. no MoE checkpoint)
@dataclass @dataclass
+243 -76
View File
@@ -9,20 +9,55 @@ streaming compute.
For each reduced artifact it writes ``<out>/<family>/<id>.pdf`` plus a sibling For each reduced artifact it writes ``<out>/<family>/<id>.pdf`` plus a sibling
``<id>.yaml`` (per-plot gallery metadata) and a per-family ``metadata.yaml``. ``<id>.yaml`` (per-plot gallery metadata) and a per-family ``metadata.yaml``.
Optionally runs ``gallery generate`` to build the static HTML site. Optionally runs ``gallery generate`` to build the static HTML site.
Every rollout series gets a stable color via ``ps.get_color(i)``, ``i`` being
its position in ``payload["series"]`` that position is fixed by the run's
YAML/``--label`` order (threaded unchanged from ``condor.RunMeta.rollouts``
through every ``PlotSpec``), so a given rollout keeps the same color across
every plot in a run. The reference, where a plot has one, always draws in one
fixed, distinct style (dark ink, dashed) instead of taking a slot in that
cycle.
""" """
from __future__ import annotations from __future__ import annotations
import dataclasses
import subprocess import subprocess
from pathlib import Path from pathlib import Path
import numpy as np import numpy as np
import plotstyle as ps import plotstyle as ps
from matplotlib.colors import LogNorm
import yaml import yaml
from giant.analysis.reduced import Reduced from giant.analysis.reduced import Reduced
_SERIES_LABELS = {"rollout": "rollout", "reference": "reference (Geant4)"} _REFERENCE_LABEL = "reference (Geant4)"
_TEX_ESCAPE_MAP = {
"\\": r"\textbackslash{}",
"%": r"\%",
"&": r"\&",
"#": r"\#",
"$": r"\$",
"_": r"\_",
"{": r"\{",
"}": r"\}",
}
def _tex_escape(text: str) -> str:
"""Escape characters LaTeX treats specially in catalog-authored title/xlabel
text (e.g. a literal ``%`` in a "90% of deposited energy" title, which
``usetex`` otherwise reads as a comment marker and aborts the whole figure
see gitea #81). A single pass over the *original* characters, so the
backslashes an escape itself introduces (e.g. ``\textbackslash{}``) are
never re-escaped."""
return "".join(_TEX_ESCAPE_MAP.get(c, c) for c in text)
def _ref_color() -> str:
return ps.colors.INK["primary"]
def _density(counts: list[int] | np.ndarray, edges: np.ndarray) -> np.ndarray: def _density(counts: list[int] | np.ndarray, edges: np.ndarray) -> np.ndarray:
@@ -33,10 +68,13 @@ def _density(counts: list[int] | np.ndarray, edges: np.ndarray) -> np.ndarray:
return counts / (total * (edges[1] - edges[0])) return counts / (total * (edges[1] - edges[0]))
def _overlay(ax, edges: np.ndarray, series: dict[str, list], log_y: bool) -> None: def _overlay(ax, edges: np.ndarray, payload: dict, log_y: bool) -> None:
for key in ("reference", "rollout"): if "reference" in payload:
if key in series: ax.stairs(
ax.stairs(_density(series[key], edges), edges, label=_SERIES_LABELS[key]) _density(payload["reference"], edges), edges, label=_REFERENCE_LABEL, color=_ref_color(), linestyle="--"
)
for i, (name, counts) in enumerate(payload.get("series", {}).items()):
ax.stairs(_density(counts, edges), edges, label=name, color=ps.get_color(i))
if log_y: if log_y:
ax.set_yscale("log") ax.set_yscale("log")
@@ -47,8 +85,8 @@ def _router_summary(router_cfg: dict) -> str:
return f"{router_cfg.get('type', '?')}×{router_cfg.get('n_experts', '?')}" return f"{router_cfg.get('type', '?')}×{router_cfg.get('n_experts', '?')}"
def _figure_params_v2(mc: dict, run_meta: dict) -> dict: def _figure_params_v2(mc: dict, meta: dict) -> dict:
"""`_figure_params` for a new-shape (nested) `model_config` — has a """`_figure_params_single` for a new-shape (nested) `model_config` — has a
`stage1_model` key. Reports stage 1's architecture (the headline `stage1_model` key. Reports stage 1's architecture (the headline
generator); stage 2's generator is only added (`mode_s2`) when it generator); stage 2's generator is only added (`mode_s2`) when it
differs from stage 1's, since a mixed run (the `stage1=flow` + differs from stage 1's, since a mixed run (the `stage1=flow` +
@@ -71,23 +109,24 @@ def _figure_params_v2(mc: dict, run_meta: dict) -> dict:
if particle_type is not None: if particle_type is not None:
params["conditioning"] = particle_type params["conditioning"] = particle_type
params["router"] = _router_summary(s1.get("router") or {}) params["router"] = _router_summary(s1.get("router") or {})
if run_meta.get("training_epoch") is not None: if meta.get("training_epoch") is not None:
params["epoch"] = run_meta["training_epoch"] params["epoch"] = meta["training_epoch"]
if run_meta.get("best_val_loss") is not None: if meta.get("best_val_loss") is not None:
params["best_val_loss"] = round(run_meta["best_val_loss"], 4) params["best_val_loss"] = round(meta["best_val_loss"], 4)
if mode == "wgan": if mode == "wgan":
noise_dim = (s1.get("wgan") or {}).get("noise_dim") noise_dim = (s1.get("wgan") or {}).get("noise_dim")
if noise_dim is not None: if noise_dim is not None:
params["noise_dim"] = noise_dim params["noise_dim"] = noise_dim
elif run_meta.get("steps") is not None: elif meta.get("steps") is not None:
params["steps"] = run_meta["steps"] params["steps"] = meta["steps"]
return params return params
def _figure_params(run_meta: dict) -> dict: def _figure_params_single(meta: dict) -> dict:
"""Curated run identity for the figure subtitle (``new_figure(params=...)``). """Curated run identity for the figure subtitle (``new_figure(params=...)``),
for exactly one rollout's ``plot_meta``.
``run_meta``/each plot's own ``<id>.yaml`` (see ``_plot_metadata``) already ``meta``/each plot's own ``<id>.yaml`` (see ``_plot_metadata``) already
carry every threaded model/training/rollout/dataset parameter for carry every threaded model/training/rollout/dataset parameter for
after-the-fact lookup this picks only the handful that matter for after-the-fact lookup this picks only the handful that matter for
telling figures apart at a glance while flipping through a gallery, since telling figures apart at a glance while flipping through a gallery, since
@@ -99,9 +138,9 @@ def _figure_params(run_meta: dict) -> dict:
Handles both a v0.2 checkpoint's flat ``model_config`` and a v0.3.0 Handles both a v0.2 checkpoint's flat ``model_config`` and a v0.3.0
nested one (has a ``stage1_model`` key see ``_figure_params_v2``). nested one (has a ``stage1_model`` key see ``_figure_params_v2``).
""" """
mc = run_meta.get("model_config") or {} mc = meta.get("model_config") or {}
if "stage1_model" in mc: if "stage1_model" in mc:
return _figure_params_v2(mc, run_meta) return _figure_params_v2(mc, meta)
mode = mc.get("mode") mode = mc.get("mode")
params: dict = {} params: dict = {}
@@ -114,18 +153,35 @@ def _figure_params(run_meta: dict) -> dict:
if mc.get("conditioning") is not None: if mc.get("conditioning") is not None:
params["conditioning"] = mc["conditioning"] params["conditioning"] = mc["conditioning"]
params["router"] = _router_summary(mc.get("router") or {}) params["router"] = _router_summary(mc.get("router") or {})
if run_meta.get("training_epoch") is not None: if meta.get("training_epoch") is not None:
params["epoch"] = run_meta["training_epoch"] params["epoch"] = meta["training_epoch"]
if run_meta.get("best_val_loss") is not None: if meta.get("best_val_loss") is not None:
params["best_val_loss"] = round(run_meta["best_val_loss"], 4) params["best_val_loss"] = round(meta["best_val_loss"], 4)
if mode == "wgan": if mode == "wgan":
if mc.get("noise_dim") is not None: if mc.get("noise_dim") is not None:
params["noise_dim"] = mc["noise_dim"] params["noise_dim"] = mc["noise_dim"]
elif run_meta.get("steps") is not None: elif meta.get("steps") is not None:
params["steps"] = run_meta["steps"] params["steps"] = meta["steps"]
return params return params
def _figure_params(run_meta: dict) -> dict:
"""Curated run identity for the figure subtitle.
A single-rollout run reuses that rollout's ``plot_meta`` (same curated
model/training/rollout subset as always see ``_figure_params_single``);
a multi-rollout run instead names the series being compared, since no
single ``model_config`` applies to the figure as a whole (each plot's own
gallery YAML still carries every rollout's full ``plot_meta`` for
after-the-fact lookup, via ``_plot_metadata``).
"""
rollouts = run_meta.get("rollouts") or {}
if len(rollouts) == 1:
((_, meta),) = rollouts.items()
return _figure_params_single(meta)
return {"rollouts": ", ".join(rollouts)} if rollouts else {}
def _render_overlay(r: Reduced, params: dict): def _render_overlay(r: Reduced, params: dict):
edges = np.asarray(r.payload["edges"]) edges = np.asarray(r.payload["edges"])
fig, ax = ps.new_figure("thesis-single", title=r.title, params=params) fig, ax = ps.new_figure("thesis-single", title=r.title, params=params)
@@ -139,7 +195,8 @@ def _render_overlay(r: Reduced, params: dict):
def _render_single(r: Reduced, params: dict): def _render_single(r: Reduced, params: dict):
edges = np.asarray(r.payload["edges"]) edges = np.asarray(r.payload["edges"])
fig, ax = ps.new_figure("thesis-single", title=r.title, params=params) fig, ax = ps.new_figure("thesis-single", title=r.title, params=params)
ax.stairs(_density(r.payload["rollout"], edges), edges, label=_SERIES_LABELS["rollout"]) for i, (name, counts) in enumerate(r.payload.get("series", {}).items()):
ax.stairs(_density(counts, edges), edges, label=name, color=ps.get_color(i))
if r.payload.get("log_y"): if r.payload.get("log_y"):
ax.set_yscale("log") ax.set_yscale("log")
if r.payload.get("log_x"): if r.payload.get("log_x"):
@@ -181,11 +238,17 @@ def _render_profile(r: Reduced, params: dict):
edges = np.asarray(r.payload["edges"]) edges = np.asarray(r.payload["edges"])
centers = 0.5 * (edges[:-1] + edges[1:]) centers = 0.5 * (edges[:-1] + edges[1:])
fig, ax = ps.new_figure("thesis-single", title=r.title, params=params) fig, ax = ps.new_figure("thesis-single", title=r.title, params=params)
for key in ("reference", "rollout"): if "reference" in r.payload:
mean = np.asarray(r.payload[f"{key}_mean"]) ref = r.payload["reference"]
std = np.asarray(r.payload[f"{key}_std"]) mean, std = np.asarray(ref["mean"]), np.asarray(ref["std"])
(line,) = ax.plot(centers, mean, label=_SERIES_LABELS[key]) color = _ref_color()
ax.fill_between(centers, mean - std, mean + std, alpha=0.2, color=line.get_color()) ax.plot(centers, mean, label=_REFERENCE_LABEL, color=color, linestyle="--")
ax.fill_between(centers, mean - std, mean + std, alpha=0.2, color=color)
for i, (name, side) in enumerate(r.payload.get("series", {}).items()):
mean, std = np.asarray(side["mean"]), np.asarray(side["std"])
color = ps.get_color(i)
ax.plot(centers, mean, label=name, color=color)
ax.fill_between(centers, mean - std, mean + std, alpha=0.2, color=color)
ax.set_xlabel(r.xlabel) ax.set_xlabel(r.xlabel)
ax.set_ylabel(r.payload.get("ylabel", "mean deposited energy [MeV]")) ax.set_ylabel(r.payload.get("ylabel", "mean deposited energy [MeV]"))
ps.style_legend(ax, title="source") ps.style_legend(ax, title="source")
@@ -195,10 +258,19 @@ def _render_profile(r: Reduced, params: dict):
def _render_bar(r: Reduced, params: dict): def _render_bar(r: Reduced, params: dict):
labels = r.payload["labels"] labels = r.payload["labels"]
x = np.arange(len(labels)) x = np.arange(len(labels))
width = 0.4 series = r.payload.get("series", {})
has_ref = "reference" in r.payload
n_bars = len(series) + (1 if has_ref else 0)
width = 0.8 / max(n_bars, 1)
offsets = np.linspace(-0.4 + width / 2, 0.4 - width / 2, n_bars)
fig, ax = ps.new_figure("thesis-single", title=r.title, params=params) fig, ax = ps.new_figure("thesis-single", title=r.title, params=params)
ax.bar(x - width / 2, r.payload["reference"], width, label=_SERIES_LABELS["reference"]) idx = 0
ax.bar(x + width / 2, r.payload["rollout"], width, label=_SERIES_LABELS["rollout"]) if has_ref:
ax.bar(x + offsets[idx], r.payload["reference"], width, label=_REFERENCE_LABEL, color=_ref_color())
idx += 1
for i, (name, vals) in enumerate(series.items()):
ax.bar(x + offsets[idx], vals, width, label=name, color=ps.get_color(i))
idx += 1
ax.set_xticks(x) ax.set_xticks(x)
ax.set_xticklabels(labels, rotation=45, ha="right") ax.set_xticklabels(labels, rotation=45, ha="right")
ax.set_ylabel(r.payload.get("ylabel", "value")) ax.set_ylabel(r.payload.get("ylabel", "value"))
@@ -207,56 +279,144 @@ def _render_bar(r: Reduced, params: dict):
def _render_router_gating(r: Reduced, params: dict): def _render_router_gating(r: Reduced, params: dict):
n_experts = r.payload["n_experts"] series = r.payload.get("series", {})
names = list(series)
log_x = r.payload.get("log_x", False) log_x = r.payload.get("log_x", False)
fig, axes = ps.new_figure("slide-16x9", title=r.title, params=params, nrows=1, ncols=2, squeeze=False) fig, axes = ps.new_figure("slide-16x9", title=r.title, params=params, nrows=len(names), ncols=2, squeeze=False)
flat = axes.ravel() for row, name in enumerate(names):
for ax, key in zip(flat, ("rollout", "reference")): entry = series[name]
side = r.payload.get(key, {}) n_experts = entry["n_experts"]
centers = np.asarray(side.get("centers", [])) for col, key in enumerate(("rollout", "reference")):
means = np.asarray(side.get("means", [])) ax = axes[row, col]
if len(centers) and means.size: side = entry.get(key, {})
cum = np.zeros(len(centers)) centers = np.asarray(side.get("centers", []))
for i in range(n_experts): means = np.asarray(side.get("means", []))
ax.fill_between(centers, cum, cum + means[:, i], alpha=0.7, label=f"expert {i}") if len(centers) and means.size:
cum = cum + means[:, i] cum = np.zeros(len(centers))
if log_x: for i in range(n_experts):
ax.set_xscale("log") ax.fill_between(centers, cum, cum + means[:, i], alpha=0.7, label=f"expert {i}")
ax.set_ylim(0, 1) cum = cum + means[:, i]
ax.set_title(_SERIES_LABELS[key], fontsize=8) if log_x:
ax.set_xlabel(r.xlabel) ax.set_xscale("log")
flat[0].set_ylabel("mean gate weight") ax.set_ylim(0, 1)
ps.style_legend(flat[0], title=f"{r.payload.get('router_type', '')} router") panel_label = _REFERENCE_LABEL if key == "reference" else "rollout"
ax.set_title(f"{name}{panel_label}", fontsize=8)
if row == len(names) - 1:
ax.set_xlabel(r.xlabel)
axes[row, 0].set_ylabel("mean gate weight")
if names:
ps.style_legend(axes[0, 0], title=f"{series[names[0]]['router_type']} router")
return fig return fig
def _render_router_share(r: Reduced, params: dict): def _render_router_share(r: Reduced, params: dict):
categories = r.payload["categories"] series = r.payload.get("series", {})
n_experts = r.payload["n_experts"] names = list(series)
x = np.arange(len(categories)) present: tuple[str, ...] = ("rollout", "reference")
present = [k for k in ("rollout", "reference") if k in r.payload] if names:
present = tuple(k for k in ("rollout", "reference") if k in series[names[0]])
ncols = max(len(present), 1)
fig, axes = ps.new_figure("slide-16x9", title=r.title, params=params, nrows=len(names), ncols=ncols, squeeze=False)
for row, name in enumerate(names):
entry = series[name]
n_experts = entry["n_experts"]
cats = entry["categories"]
x = np.arange(len(cats))
for col, key in enumerate(present):
ax = axes[row, col]
side = entry.get(key)
if side is not None:
shares = np.array([side[c] for c in cats]) # (n_cat, n_experts)
bottom = np.zeros(len(cats))
for i in range(n_experts):
ax.bar(x, shares[:, i], bottom=bottom, label=f"expert {i}")
bottom += shares[:, i]
ax.set_xticks(x)
ax.set_xticklabels(cats, rotation=45, ha="right")
ax.set_ylim(0, 1)
panel_label = _REFERENCE_LABEL if key == "reference" else "rollout"
ax.set_title(f"{name}{panel_label}", fontsize=8)
axes[row, 0].set_ylabel("share of rows dispatched to expert")
if names:
ps.style_legend(axes[0, 0], title=f"{series[names[0]]['router_type']} router")
return fig
def _render_router_specialization(r: Reduced, params: dict):
fig, ax = ps.new_figure("thesis-single", title=r.title, params=params)
series = r.payload.get("series", {})
chance_levels: set[float] = set()
for i, (name, entry) in enumerate(series.items()):
color = ps.get_color(i)
if entry.get("chance_level") is not None:
chance_levels.add(entry["chance_level"])
for key, linestyle, label in (
("rollout", "-", name),
("reference", "--", f"{name} ({_REFERENCE_LABEL})"),
):
side = entry.get(key)
if side and side["centers"]:
ax.plot(
side["centers"],
side["score"],
label=label,
color=color,
linestyle=linestyle,
marker="o",
markersize=3,
)
for lvl in sorted(chance_levels):
ax.axhline(lvl, linestyle=":", color="gray")
if r.payload.get("log_x"):
ax.set_xscale("log")
ax.set_ylim(0, 1)
ax.set_xlabel(r.xlabel)
ax.set_ylabel("max gate weight")
ps.style_legend(ax, title="router")
return fig
def _render_heatmap(r: Reduced, params: dict):
series = dict(r.payload["series"])
row_labels = r.payload["row_labels"]
col_labels = r.payload["col_labels"]
# A heatmap-shaped plot is one matrix per rollout, so the reference (when the
# comparison has one — the distance scorecard doesn't) becomes one more panel
# rather than another line.
if r.payload.get("reference") is not None:
series["reference"] = r.payload["reference"]
names = list(series)
norm = LogNorm(vmin=1) if r.payload.get("log_color") else None
fig, axes = ps.new_figure( fig, axes = ps.new_figure(
"slide-16x9", "slide-16x9" if len(names) > 1 else "thesis-single",
title=r.title, title=r.title,
params=params, params=params,
nrows=1, nrows=1,
ncols=len(present), ncols=len(names),
squeeze=False, squeeze=False,
) )
flat = axes.ravel() flat = axes.ravel()
for ax, key in zip(flat, present): im = None
side = r.payload[key] for ax, name in zip(flat, names):
shares = np.array([side[c] for c in categories]) # (n_cat, n_experts) mat = np.asarray(series[name], dtype=float)
bottom = np.zeros(len(categories)) im = ax.imshow(
for i in range(n_experts): mat,
ax.bar(x, shares[:, i], bottom=bottom, label=f"expert {i}") origin="upper",
bottom += shares[:, i] aspect="auto",
ax.set_xticks(x) cmap=r.payload.get("cmap", "viridis"),
ax.set_xticklabels(categories, rotation=45, ha="right") norm=norm,
ax.set_ylim(0, 1) vmin=None if norm else r.payload.get("vmin"),
ax.set_title(_SERIES_LABELS[key], fontsize=8) vmax=None if norm else r.payload.get("vmax"),
flat[0].set_ylabel("share of rows dispatched to expert") )
ps.style_legend(flat[0], title=f"{r.payload.get('router_type', '')} router") ax.set_xticks(range(len(col_labels)))
ax.set_xticklabels(col_labels, rotation=45, ha="right")
ax.set_yticks(range(len(row_labels)))
ax.set_yticklabels(row_labels)
ax.set_xlabel(r.xlabel)
if len(names) > 1:
ax.set_title(name, fontsize=8)
flat[0].set_ylabel(r.payload.get("ylabel", ""))
fig.colorbar(im, ax=list(flat), label=r.payload.get("cbar_label", "value"))
return fig return fig
@@ -284,13 +444,21 @@ _RENDERERS = {
"bar": _render_bar, "bar": _render_bar,
"router_gating": _render_router_gating, "router_gating": _render_router_gating,
"router_share": _render_router_share, "router_share": _render_router_share,
"router_specialization": _render_router_specialization,
"heatmap": _render_heatmap,
"unavailable": _render_unavailable, "unavailable": _render_unavailable,
} }
def render(r: Reduced, run_meta: dict | None = None): def render(r: Reduced, run_meta: dict | None = None):
"""Build the matplotlib figure for one reduced artifact (dispatch on kind).""" """Build the matplotlib figure for one reduced artifact (dispatch on kind).
return _RENDERERS[r.kind](r, _figure_params(run_meta or {}))
``title``/``xlabel`` are LaTeX-escaped here, at the one point every kind's
renderer draws them from ``_plot_metadata`` deliberately keeps using the
unescaped ``r`` for the gallery YAML, which isn't LaTeX.
"""
escaped = dataclasses.replace(r, title=_tex_escape(r.title), xlabel=_tex_escape(r.xlabel))
return _RENDERERS[r.kind](escaped, _figure_params(run_meta or {}))
def _plot_metadata(r: Reduced, run_meta: dict) -> dict: def _plot_metadata(r: Reduced, run_meta: dict) -> dict:
@@ -349,7 +517,7 @@ def render_all(
yaml.safe_dump( yaml.safe_dump(
{ {
"title": run_meta.get("title", "GIANT rollout analysis"), "title": run_meta.get("title", "GIANT rollout analysis"),
"description": "Autoregressive rollout compared against held-out Geant4 reference steps.", "description": "Autoregressive rollout(s) compared against a held-out Geant4 reference steps file.",
"experiment": "GIANT", "experiment": "GIANT",
"parameters": {k: v for k, v in run_meta.items() if k != "title"}, "parameters": {k: v for k, v in run_meta.items() if k != "title"},
}, },
@@ -382,8 +550,7 @@ def render_run(run_dir: str | Path, *, run_gallery: bool = False) -> list[Path]:
meta = RunMeta.load(run_dir / "run_meta.json") meta = RunMeta.load(run_dir / "run_meta.json")
run_meta = { run_meta = {
"title": meta.title, "title": meta.title,
"rollout": meta.rollout,
"reference": meta.reference, "reference": meta.reference,
**meta.plot_meta, "rollouts": {ro["name"]: ro["plot_meta"] for ro in meta.rollouts},
} }
return render_all(run_dir / "reduced", run_dir / "plots", run_meta, run_gallery=run_gallery) return render_all(run_dir / "reduced", run_dir / "plots", run_meta, run_gallery=run_gallery)
+124 -53
View File
@@ -35,6 +35,7 @@ from giant.analysis.reduced import Reduced
if TYPE_CHECKING: if TYPE_CHECKING:
import torch import torch
from giant.analysis.sources import RolloutSide
from giant.data.transforms import Normalizer from giant.data.transforms import Normalizer
_SAMPLE_ROWS = 200_000 _SAMPLE_ROWS = 200_000
@@ -203,6 +204,7 @@ _TITLES = {
"router_gating": "Router gating (mixture-of-experts decision boundaries)", "router_gating": "Router gating (mixture-of-experts decision boundaries)",
"router_share_by_pdg": "Router expert share by particle species", "router_share_by_pdg": "Router expert share by particle species",
"router_share_by_process": "Router expert share by physics process", "router_share_by_process": "Router expert share by physics process",
"router_specialization": "Router specialization score vs energy (max gate weight)",
} }
@@ -217,51 +219,103 @@ def _unavailable(spec_id: str) -> Reduced:
) )
def compute_router_gating( def _gating_entry(checkpoint: str | Path | None, r_phys: pl.LazyFrame, t_phys: pl.LazyFrame, seed: int) -> dict | None:
checkpoint: str | Path | None, """One rollout's ``router_gating`` panel data, or ``None`` if not a MoE checkpoint."""
r_phys: pl.LazyFrame,
t_phys: pl.LazyFrame,
seed: int = 0,
) -> Reduced:
"""`Reduced` for the router-gating figure, or an explanatory note if n/a."""
handle = load_router(checkpoint) if checkpoint else None handle = load_router(checkpoint) if checkpoint else None
if handle is None: if handle is None:
return _unavailable("router_gating") return None
sides: dict[str, dict] = {} sides: dict[str, dict] = {}
for name, lf in (("rollout", r_phys), ("reference", t_phys)): for name, lf in (("rollout", r_phys), ("reference", t_phys)):
df = _subsample(lf, _SAMPLE_ROWS, seed) df = _subsample(lf, _SAMPLE_ROWS, seed)
df, gate = _gate_for_df(handle, df) df, gate = _gate_for_df(handle, df)
x = df["pre_E"].to_numpy() x = df["pre_E"].to_numpy()
sides[name] = _quantile_bins(x, gate, _N_BINS) if len(x) else {"centers": [], "means": []} sides[name] = _quantile_bins(x, gate, _N_BINS) if len(x) else {"centers": [], "means": []}
return {"router_type": handle.router_type, "n_experts": handle.router.n_experts, **sides}
def compute_router_gating(rollouts: dict[str, "RolloutSide"], t_phys: pl.LazyFrame, seed: int = 0) -> Reduced:
"""`Reduced` for the router-gating figure: one panel-pair per rollout with
an enabled MoE router, or an explanatory note if none of them have one."""
series = {}
for name, rs in rollouts.items():
entry = _gating_entry(rs.checkpoint, rs.phys, t_phys, seed)
if entry is not None:
series[name] = entry
if not series:
return _unavailable("router_gating")
return Reduced( return Reduced(
id="router_gating", id="router_gating",
family="model", family="model",
kind="router_gating", kind="router_gating",
title=_TITLES["router_gating"], title=_TITLES["router_gating"],
xlabel="pre-step energy [MeV]", xlabel="pre-step energy [MeV]",
payload={ payload={"series": series, "log_x": True},
"router_type": handle.router_type,
"n_experts": handle.router.n_experts,
"log_x": True,
**sides,
},
) )
def compute_router_share_by_pdg( def _specialization_entry(
checkpoint: str | Path | None, checkpoint: str | Path | None, r_phys: pl.LazyFrame, t_phys: pl.LazyFrame, seed: int
r_phys: pl.LazyFrame, ) -> dict | None:
t_phys: pl.LazyFrame, """One rollout's ``router_specialization`` curve data, or ``None`` if not a MoE checkpoint.
top_pdgs: list[int],
seed: int = 0, Scalar specialization trend: max gate weight vs energy, per side.
) -> Reduced: Summarizes `router_gating`'s full per-expert stacked area into one curve —
"""Stacked-bar share of each particle species dispatched to each expert.""" the routing plan's own "how sharp is the boundary here" number (1/n_experts
= uniform/no specialization, 1.0 = one expert fully owns that energy). Same
quantile energy bins as `router_gating` (`_quantile_bins`), so this is
directly comparable to that plot's ceiling described in the roadmap's MoE
writeup.
"""
handle = load_router(checkpoint) if checkpoint else None handle = load_router(checkpoint) if checkpoint else None
if handle is None: if handle is None:
return _unavailable("router_share_by_pdg") return None
sides: dict[str, dict] = {}
for name, lf in (("rollout", r_phys), ("reference", t_phys)):
df = _subsample(lf, _SAMPLE_ROWS, seed)
df, gate = _gate_for_df(handle, df)
x = df["pre_E"].to_numpy()
if len(x):
binned = _quantile_bins(x, gate, _N_BINS)
means = np.asarray(binned["means"])
score = means.max(axis=1).tolist() if means.size else []
sides[name] = {"centers": binned["centers"], "score": score}
else:
sides[name] = {"centers": [], "score": []}
return {
"router_type": handle.router_type,
"n_experts": handle.router.n_experts,
"chance_level": 1.0 / handle.router.n_experts,
**sides,
}
def compute_router_specialization(rollouts: dict[str, "RolloutSide"], t_phys: pl.LazyFrame, seed: int = 0) -> Reduced:
"""`Reduced` for the router-specialization figure, one curve per rollout with
an enabled MoE router (see `_specialization_entry`)."""
series = {}
for name, rs in rollouts.items():
entry = _specialization_entry(rs.checkpoint, rs.phys, t_phys, seed)
if entry is not None:
series[name] = entry
if not series:
return _unavailable("router_specialization")
return Reduced(
id="router_specialization",
family="model",
kind="router_specialization",
title=_TITLES["router_specialization"],
xlabel="pre-step energy [MeV]",
payload={"series": series, "log_x": True},
)
def _share_by_pdg_entry(
checkpoint: str | Path | None, r_phys: pl.LazyFrame, t_phys: pl.LazyFrame, top_pdgs: list[int], seed: int
) -> dict | None:
"""One rollout's ``router_share_by_pdg`` panel-pair data, or ``None`` if not a MoE checkpoint."""
handle = load_router(checkpoint) if checkpoint else None
if handle is None:
return None
labels = [pdg_label(p) for p in top_pdgs] labels = [pdg_label(p) for p in top_pdgs]
sides: dict[str, dict] = {} sides: dict[str, dict] = {}
for name, lf in (("rollout", r_phys), ("reference", t_phys)): for name, lf in (("rollout", r_phys), ("reference", t_phys)):
@@ -273,41 +327,36 @@ def compute_router_share_by_pdg(
else: else:
shares = {str(p): [0.0] * handle.router.n_experts for p in top_pdgs} shares = {str(p): [0.0] * handle.router.n_experts for p in top_pdgs}
sides[name] = {labels[i]: shares[str(p)] for i, p in enumerate(top_pdgs)} sides[name] = {labels[i]: shares[str(p)] for i, p in enumerate(top_pdgs)}
return {"router_type": handle.router_type, "n_experts": handle.router.n_experts, "categories": labels, **sides}
def compute_router_share_by_pdg(
rollouts: dict[str, "RolloutSide"], t_phys: pl.LazyFrame, top_pdgs: list[int], seed: int = 0
) -> Reduced:
"""`Reduced` for the router expert-share-by-species figure, one panel-pair
per rollout with an enabled MoE router."""
series = {}
for name, rs in rollouts.items():
entry = _share_by_pdg_entry(rs.checkpoint, rs.phys, t_phys, top_pdgs, seed)
if entry is not None:
series[name] = entry
if not series:
return _unavailable("router_share_by_pdg")
return Reduced( return Reduced(
id="router_share_by_pdg", id="router_share_by_pdg",
family="model", family="model",
kind="router_share", kind="router_share",
title=_TITLES["router_share_by_pdg"], title=_TITLES["router_share_by_pdg"],
xlabel="particle species", xlabel="particle species",
payload={ payload={"series": series},
"router_type": handle.router_type,
"n_experts": handle.router.n_experts,
"categories": labels,
**sides,
},
) )
def compute_router_share_by_process( def _share_by_process_entry(checkpoint: str | Path | None, t_phys: pl.LazyFrame, seed: int, top_k: int) -> dict | None:
checkpoint: str | Path | None, """One rollout checkpoint's ``router_share_by_process`` panel data (reference-only), or ``None`` if not MoE."""
t_phys: pl.LazyFrame,
seed: int = 0,
top_k: int = _TOP_K_PROCESS,
) -> Reduced:
"""Stacked-bar share of each physics process dispatched to each expert.
Reference-only: ``process`` is the true post-step physics process a
label the rollout side has no equivalent of (see
`giant.model.network.ProcessRouter`, which predicts it from pre-step
conditioning alone, never observes it at eval time). This plot instead
checks *after the fact*, on real data, how well the router's conditioning
-based dispatch lines up with the true process.
"""
handle = load_router(checkpoint) if checkpoint else None handle = load_router(checkpoint) if checkpoint else None
if handle is None: if handle is None:
return _unavailable("router_share_by_process") return None
df = _subsample(t_phys, _SAMPLE_ROWS, seed, extra_cols=("process",)) df = _subsample(t_phys, _SAMPLE_ROWS, seed, extra_cols=("process",))
df, gate = _gate_for_df(handle, df) df, gate = _gate_for_df(handle, df)
if len(df): if len(df):
@@ -317,17 +366,39 @@ def compute_router_share_by_process(
shares = _top1_shares(df["process"].to_numpy(), idx, order, handle.router.n_experts) shares = _top1_shares(df["process"].to_numpy(), idx, order, handle.router.n_experts)
else: else:
order, shares = [], {} order, shares = [], {}
return {
"router_type": handle.router_type,
"n_experts": handle.router.n_experts,
"categories": order,
"reference": {p: shares[p] for p in order},
}
def compute_router_share_by_process(
rollouts: dict[str, "RolloutSide"], t_phys: pl.LazyFrame, seed: int = 0, top_k: int = _TOP_K_PROCESS
) -> Reduced:
"""Stacked-bar share of each physics process dispatched to each expert, one
panel per rollout checkpoint with an enabled MoE router.
Reference-only: ``process`` is the true post-step physics process a
label the rollout side has no equivalent of (see
`giant.model.network.ProcessRouter`, which predicts it from pre-step
conditioning alone, never observes it at eval time). This plot instead
checks *after the fact*, on real data, how well each checkpoint's router
-based dispatch lines up with the true process.
"""
series = {}
for name, rs in rollouts.items():
entry = _share_by_process_entry(rs.checkpoint, t_phys, seed, top_k)
if entry is not None:
series[name] = entry
if not series:
return _unavailable("router_share_by_process")
return Reduced( return Reduced(
id="router_share_by_process", id="router_share_by_process",
family="model", family="model",
kind="router_share", kind="router_share",
title=_TITLES["router_share_by_process"], title=_TITLES["router_share_by_process"],
xlabel="physics process", xlabel="physics process",
payload={ payload={"series": series},
"router_type": handle.router_type,
"n_experts": handle.router.n_experts,
"categories": order,
"reference": {p: shares[p] for p in order},
},
) )
+3 -3
View File
@@ -6,8 +6,8 @@ streaming `group_by` pass(es) over the chunk (see `catalog.py`/`reduce.py`).
`_COST_MODEL` below is ``spec_id -> (intercept_s, seconds_per_row)``. `_COST_MODEL` below is ``spec_id -> (intercept_s, seconds_per_row)``.
``n_rows`` is the combined rollout+reference row count of the job's input: ``n_rows`` is the combined rollout+reference row count of the job's input:
the chunk's row count for `chunkable=True` specs, the whole dataset's for the the chunk's row count for `chunkable=True` specs, the whole dataset's for the
three `chunkable=False` router specs (they always run as a single job `chunkable=False` router specs in `_ROUTER_IDS` (they always run as a single
regardless of chunk count). job regardless of chunk count).
Calibrated 2026-07-27 from real HTCondor timings (`condor_history` Calibrated 2026-07-27 from real HTCondor timings (`condor_history`
``RemoteWallClockTime``) of a production run: prediction ``563f5ee3`` ``RemoteWallClockTime``) of a production run: prediction ``563f5ee3``
@@ -54,7 +54,7 @@ _FIXED_OVERHEAD_S = 60.0
# scan. Calibrated from the 3 real router jobs' observed wall times (119, 66, # scan. Calibrated from the 3 real router jobs' observed wall times (119, 66,
# 124s) — max minus _FIXED_OVERHEAD_S, on top of it. # 124s) — max minus _FIXED_OVERHEAD_S, on top of it.
_ROUTER_FIXED_S = 64.0 _ROUTER_FIXED_S = 64.0
_ROUTER_IDS = frozenset({"router_gating", "router_share_by_pdg", "router_share_by_process"}) _ROUTER_IDS = frozenset({"router_gating", "router_share_by_pdg", "router_share_by_process", "router_specialization"})
# Conservative fallback for any catalog id not in _COST_MODEL (e.g. a plot # Conservative fallback for any catalog id not in _COST_MODEL (e.g. a plot
# added after the last calibration run) — the most expensive fitted per-row # added after the last calibration run) — the most expensive fitted per-row
+73 -4
View File
@@ -1,7 +1,12 @@
"""Canonical world-frame LazyFrame builders for the two sides of a comparison. """Canonical world-frame LazyFrame builders for the two kinds of comparison input.
The analysis compares one autoregressive ``giant rollout`` (the *generated* side) The analysis compares one or more autoregressive ``giant rollout`` runs (the
against a raw miniCaloSim steps file (the *reference* / real side). Both carry a *generated* side one named series each, see ``RolloutSpec``) against a single
raw miniCaloSim steps file shared by all of them (the *reference* / real side).
Every rollout is the same *kind* of file regardless of how many there are, so
``Side`` stays binary: it describes a file's schema (rollout column layout +
synthetic-termination rows + per-track secondary view, vs. reference
``sec_*_list`` columns), not series identity. Both kinds carry a
**shared world-frame physical column subset** under identical names, so no **shared world-frame physical column subset** under identical names, so no
renaming or coordinate decode is needed everything is already in world-frame renaming or coordinate decode is needed everything is already in world-frame
mm / MeV: mm / MeV:
@@ -26,6 +31,7 @@ HTCondor workers that have no LaTeX toolchain.
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass
from enum import Enum from enum import Enum
from pathlib import Path from pathlib import Path
@@ -82,12 +88,44 @@ SYNTHETIC_TERMINATION_REASONS: frozenset[str] = frozenset(
class Side(str, Enum): class Side(str, Enum):
"""Which of the two comparison inputs a file is.""" """Which of the two comparison-input *kinds* a file is."""
rollout = "rollout" rollout = "rollout"
reference = "reference" reference = "reference"
@dataclass
class RolloutSpec:
"""One named rollout input, as fed to ``build_context``/``Bundle.open``.
``name`` is the series' identity throughout the rest of the pipeline (a
plot's ``payload["series"]`` key, a figure's legend label, its color)
resolved once in ``condor.load_rollout_yamls`` from ``--label`` or the
YAML stem, then threaded through unchanged. ``checkpoint`` /
``type_embedding_l1_dist`` are only used by the router/type-embedding
diagnostics (``catalog.py``'s ``chunkable=False`` specs).
"""
name: str
source: str | Path | pl.LazyFrame
checkpoint: str | None = None
type_embedding_l1_dist: dict | None = None
@dataclass
class RolloutSide:
"""One rollout's opened frames + per-checkpoint diagnostic inputs (``catalog.Bundle.rollouts`` value)."""
all: pl.LazyFrame # rollout, all rows (incl. synthetic termination rows)
phys: pl.LazyFrame # rollout, physical steps only
checkpoint: str | None = None # from the rollout YAML; router_gating only
# Diagnostic pre-aggregated at rollout time (giant.rollout.
# L1DistCollector.summary()) — from the rollout YAML, type_embedding_l1_distance
# only. Unlike checkpoint, this needs no live model: it's already a
# finished histogram, just passed through.
type_embedding_l1_dist: dict | None = None
def _check_rollout_metadata(path: Path) -> None: def _check_rollout_metadata(path: Path) -> None:
"""Raise if ``path`` carries coord metadata that isn't the rollout tag. """Raise if ``path`` carries coord metadata that isn't the rollout tag.
@@ -198,3 +236,34 @@ def secondaries(lf: pl.LazyFrame, side: Side) -> pl.LazyFrame:
pl.col("sec_dz_list").alias("sdz"), pl.col("sec_dz_list").alias("sdz"),
) )
) )
def secondaries_by_step(lf: pl.LazyFrame, side: Side) -> pl.LazyFrame:
"""One row per produced secondary, tagged with the step that produced it.
Canonical columns: ``step_key`` (an opaque struct identifying the emitting
step) and ``pdg``. ``secondaries`` deliberately drops that link; the
per-step multiplicity plots need it, so this is a separate view rather than
extra columns every other consumer would pay for.
- rollout: a secondary's birth row carries ``parent_id`` and a birth
position copied verbatim from the parent step's ``post_pos``, so
``(event_id, parent_id, pre_pos)`` identifies the emitting step exactly
no join against the (large) step frame is needed.
- reference: secondaries already live on their parent step's row, so the
row index *is* the step key. It is only ever used as a group key inside
one chunk's own aggregation, so indices repeating across chunks is
harmless.
"""
if side is Side.rollout:
return lf.filter((pl.col("generation") > 0) & (pl.col("step_no") == 0)).select(
pl.struct("event_id", "parent_id", "pre_x", "pre_y", "pre_z").alias("step_key"),
"pdg",
)
return (
lf.select("sec_pdg_list")
.with_row_index("_row")
.explode("sec_pdg_list")
.drop_nulls("sec_pdg_list")
.select(pl.struct("_row").alias("step_key"), pl.col("sec_pdg_list").cast(pl.Int64).alias("pdg"))
)
+29 -18
View File
@@ -20,26 +20,37 @@ redesign exists to fix.
from __future__ import annotations from __future__ import annotations
from typing import TYPE_CHECKING
from giant.analysis.reduced import Reduced from giant.analysis.reduced import Reduced
if TYPE_CHECKING:
from giant.analysis.sources import RolloutSide
_NOTE_NOT_APPLICABLE = ( _NOTE_NOT_APPLICABLE = (
"not applicable: this rollout's checkpoint doesn't use " "not applicable: none of these rollouts' checkpoints use "
"stage2_model.particle_type.target='embedding' (or generated no " "stage2_model.particle_type.target='embedding' (or generated no "
"secondaries), so giant rollout recorded no type_embedding_l1_dist " "secondaries), so giant rollout recorded no type_embedding_l1_dist "
"diagnostic in its YAML sidecar" "diagnostic in their YAML sidecar"
) )
def compute_type_embedding_l1_distance(l1_dist: dict | None) -> Reduced: def compute_type_embedding_l1_distance(rollouts: dict[str, "RolloutSide"]) -> Reduced:
"""`Reduced` for the type-embedding-distance figure, or an explanatory """`Reduced` for the type-embedding-distance figure: one series per rollout
note if this checkpoint never populated the diagnostic. whose checkpoint populated the diagnostic, or an explanatory note if none did.
`l1_dist`: `giant.rollout.L1DistCollector.summary()`'s dict, as recorded Each rollout's `RolloutSide.type_embedding_l1_dist` is
in the rollout YAML's `type_embedding_l1_dist` key (`Bundle. `giant.rollout.L1DistCollector.summary()`'s dict, as recorded in that
type_embedding_l1_dist`) `{"n", "mean", "std", "min", "max", rollout's YAML `type_embedding_l1_dist` key — `{"n", "mean", "std",
"hist_edges", "hist_counts"}`. "min", "max", "hist_edges", "hist_counts"}`. Every collector uses the
same fixed log-spaced edges (`L1DistCollector.__init__`'s defaults, never
overridden see `giant/cli.py`'s rollout command), so it's safe to plot
every rollout's counts against the first one's edges.
""" """
if l1_dist is None: entries = {
name: rs.type_embedding_l1_dist for name, rs in rollouts.items() if rs.type_embedding_l1_dist is not None
}
if not entries:
return Reduced( return Reduced(
id="type_embedding_l1_distance", id="type_embedding_l1_distance",
family="model", family="model",
@@ -49,6 +60,11 @@ def compute_type_embedding_l1_distance(l1_dist: dict | None) -> Reduced:
payload={"note": _NOTE_NOT_APPLICABLE}, payload={"note": _NOTE_NOT_APPLICABLE},
) )
edges = next(iter(entries.values()))["hist_edges"]
notes = [
f"{name}: n={d['n']:,} mean={d['mean']:.4g} std={d['std']:.4g} min={d['min']:.4g} max={d['max']:.4g}"
for name, d in entries.items()
]
return Reduced( return Reduced(
id="type_embedding_l1_distance", id="type_embedding_l1_distance",
family="model", family="model",
@@ -56,15 +72,10 @@ def compute_type_embedding_l1_distance(l1_dist: dict | None) -> Reduced:
title="Secondary-type embedding L1 distance (predicted vector -> nearest PDG row)", title="Secondary-type embedding L1 distance (predicted vector -> nearest PDG row)",
xlabel="L1 distance", xlabel="L1 distance",
payload={ payload={
"edges": l1_dist["hist_edges"], "edges": edges,
"rollout": l1_dist["hist_counts"], "series": {name: d["hist_counts"] for name, d in entries.items()},
"log_y": True, "log_y": True,
"log_x": True, "log_x": True,
"note": ( "note": "; ".join(notes) + "; rollout only, no reference concept for a raw pre-decode vector",
f"n={l1_dist['n']:,} mean={l1_dist['mean']:.4g} "
f"std={l1_dist['std']:.4g} min={l1_dist['min']:.4g} "
f"max={l1_dist['max']:.4g}; rollout only, no reference "
"concept for a raw pre-decode vector"
),
}, },
) )
+56 -7
View File
@@ -1556,10 +1556,23 @@ app.add_typer(analyze_app, name="analyze")
@analyze_app.command("prep") @analyze_app.command("prep")
def analyze_prep( def analyze_prep(
rollout_yaml: Annotated[ rollout_yamls: Annotated[
Path, list[Path],
typer.Argument(help="giant rollout YAML sidecar (names the rollout + reference files)"), typer.Argument(
help="giant rollout YAML sidecar(s) (names the rollout + reference files). "
"Multiple compare N rollouts against one shared reference — every YAML must "
"name the same `dataset`."
),
], ],
label: Annotated[
Optional[list[str]],
typer.Option(
"--label",
help="Series name for a rollout YAML, positionally matched to it — give none, "
'or exactly one per YAML. Defaults to the YAML stem (or "rollout" for a '
"single YAML).",
),
] = None,
run_dir: Annotated[ run_dir: Annotated[
Optional[Path], Optional[Path],
typer.Option( typer.Option(
@@ -1576,14 +1589,15 @@ def analyze_prep(
typer.Option("--chunks", help="Split each plot's data into this many event_id chunks"), typer.Option("--chunks", help="Split each plot's data into this many event_id chunks"),
] = 1, ] = 1,
) -> None: ) -> None:
"""Read the rollout YAML → shared.json + run_meta.json in the run directory.""" """Read the rollout YAML(s) → shared.json + run_meta.json in the run directory."""
from giant.analysis import prep from giant.analysis import prep
path = prep( path = prep(
rollout_yaml, rollout_yamls,
run_dir, run_dir,
n_chunks=chunks, n_chunks=chunks,
default_base=Path.cwd() / "analysis_runs", default_base=Path.cwd() / "analysis_runs",
labels=label,
n_energy_bins=n_energy_bins, n_energy_bins=n_energy_bins,
n_marginal_bins=n_marginal_bins, n_marginal_bins=n_marginal_bins,
top_k_pdg=top_k_pdg, top_k_pdg=top_k_pdg,
@@ -1644,10 +1658,44 @@ def analyze_render(
typer.echo(f"rendered {len(pdfs)} plots → {Path(run_dir) / 'plots'}") typer.echo(f"rendered {len(pdfs)} plots → {Path(run_dir) / 'plots'}")
@analyze_app.command("metrics")
def analyze_metrics(
run_dir: Annotated[Path, typer.Argument(help="Run directory containing metrics.csv (from `giant train`)")],
out_dir: Annotated[
Optional[Path],
typer.Option(
"--out",
"-o",
help="Override the output directory (default: <cwd>/analysis_runs/metrics_<run_dir name>)",
),
] = None,
) -> None:
"""Render training-progress plots (loss/lr/accuracy/grad-norm/router/wgan/throughput) from <run_dir>/metrics.csv."""
from giant.training.plots import render_metrics
paths = render_metrics(run_dir, out_dir, default_base=Path.cwd() / "analysis_runs")
typer.echo(f"rendered {len(paths)} plots -> {paths[0].parent if paths else '(nothing to render)'}")
@analyze_app.command("submit") @analyze_app.command("submit")
def analyze_submit( def analyze_submit(
rollout_yaml: Annotated[Path, typer.Argument(help="giant rollout YAML sidecar")], rollout_yamls: Annotated[
list[Path],
typer.Argument(
help="giant rollout YAML sidecar(s). Multiple compare N rollouts against one "
"shared reference — every YAML must name the same `dataset`."
),
],
accounting_group: Annotated[str, typer.Option("--accounting-group")], accounting_group: Annotated[str, typer.Option("--accounting-group")],
label: Annotated[
Optional[list[str]],
typer.Option(
"--label",
help="Series name for a rollout YAML, positionally matched to it — give none, "
'or exactly one per YAML. Defaults to the YAML stem (or "rollout" for a '
"single YAML).",
),
] = None,
run_dir: Annotated[ run_dir: Annotated[
Optional[Path], Optional[Path],
typer.Option( typer.Option(
@@ -1680,10 +1728,11 @@ def analyze_submit(
from giant.analysis import SubmitConfig, prep, write_submit from giant.analysis import SubmitConfig, prep, write_submit
path = prep( path = prep(
rollout_yaml, rollout_yamls,
run_dir, run_dir,
n_chunks=chunks, n_chunks=chunks,
default_base=Path.cwd() / "analysis_runs", default_base=Path.cwd() / "analysis_runs",
labels=label,
n_energy_bins=n_energy_bins, n_energy_bins=n_energy_bins,
n_marginal_bins=n_marginal_bins, n_marginal_bins=n_marginal_bins,
top_k_pdg=top_k_pdg, top_k_pdg=top_k_pdg,
+4
View File
@@ -202,6 +202,8 @@ def build_critics(model_config: dict) -> dict[str, nn.Module | None]:
cond_out_dim=cond_out_dim, cond_out_dim=cond_out_dim,
dropout=s1_spec.dropout, dropout=s1_spec.dropout,
stage="stage1", stage="stage1",
trunk_type=s1_spec.trunk.type,
block_conditioning=s1_spec.trunk.block_conditioning,
) )
if s2_spec.active and build_objective(s2_spec.generator).is_adversarial: if s2_spec.active and build_objective(s2_spec.generator).is_adversarial:
@@ -225,6 +227,8 @@ def build_critics(model_config: dict) -> dict[str, nn.Module | None]:
dropout=s2_spec.dropout, dropout=s2_spec.dropout,
stage="stage2", stage="stage2",
context_dim=s2_spec.context_dim, context_dim=s2_spec.context_dim,
trunk_type=s2_spec.trunk.type,
block_conditioning=s2_spec.trunk.block_conditioning,
) )
return result return result
+57 -32
View File
@@ -8,7 +8,7 @@ from giant.config import ConditioningAxisConfig, HeadConfig, ParticleTypeConfig
from giant.constants import CONT_SLOT_DIM, K_MAX, PARTICLE_PHYS_DIM, SEC_DIM, SEC_SLOT_DIM, X_DIM from giant.constants import CONT_SLOT_DIM, K_MAX, PARTICLE_PHYS_DIM, SEC_DIM, SEC_SLOT_DIM, X_DIM
from giant.model.encoders import ConditionEncoder from giant.model.encoders import ConditionEncoder
from giant.model.history import HistoryEncoder, build_history from giant.model.history import HistoryEncoder, build_history
from giant.model.layers import ContextAdapter, ResBlock, SinusoidalEmbedding, build_mlp_head from giant.model.layers import ContextAdapter, SinusoidalEmbedding, build_mlp_head
from giant.model.objectives import build_objective from giant.model.objectives import build_objective
from giant.model.routers import Router from giant.model.routers import Router
from giant.model.trunks import build_trunk from giant.model.trunks import build_trunk
@@ -188,6 +188,26 @@ class StageModel(nn.Module):
hidden = max(1, round(hidden_dim * head_cfg.hidden_ratio)) hidden = max(1, round(hidden_dim * head_cfg.hidden_ratio))
self.stop_head = build_mlp_head(cond_out_dim, 1, hidden, head_cfg.depth) self.stop_head = build_mlp_head(cond_out_dim, 1, hidden, head_cfg.depth)
def _build_context_fusion(self, x_dim: int, context_dim: int, cond_out_dim: int) -> None:
"""Builds `self.context_adapter`/`self.fuse` — the stage-2-style
context-fusion pattern (project the previous stage's outcome down to
`context_dim` via `ContextAdapter`, concat onto the base conditioning,
project back to `cond_out_dim`) shared by `Stage2OneShot` and a
`stage="stage2"` `CriticModel` (gitea #57). Call from a subclass's
`__init__` before using `_cond_embed`."""
self.context_adapter = ContextAdapter(x_dim, context_dim)
self.fuse = nn.Sequential(
nn.Linear(cond_out_dim + context_dim, cond_out_dim),
nn.SiLU(),
)
def _cond_embed(self, cond_cont: torch.Tensor, cond_cat: torch.Tensor, stage1_out: torch.Tensor) -> torch.Tensor:
"""Fuses base conditioning with the previous stage's outcome — pairs
with `_build_context_fusion`."""
base = self.cond_enc(cond_cont, cond_cat)
ctx = self.context_adapter(stage1_out)
return self.fuse(torch.cat([base, ctx], dim=-1))
def _require_n_sec_head(self) -> None: def _require_n_sec_head(self) -> None:
if self.n_sec_head is None: if self.n_sec_head is None:
raise RuntimeError( raise RuntimeError(
@@ -363,11 +383,7 @@ class Stage2OneShot(StageModel):
particle_type_cfg=particle_type_cfg, particle_type_cfg=particle_type_cfg,
cond_enc=cond_enc, cond_enc=cond_enc,
) )
self.context_adapter = ContextAdapter(x_dim, context_dim) self._build_context_fusion(x_dim, context_dim, cond_out_dim)
self.fuse = nn.Sequential(
nn.Linear(cond_out_dim + context_dim, cond_out_dim),
nn.SiLU(),
)
target = self.particle_type_cfg.target target = self.particle_type_cfg.target
type_head_out_dim = None if target == "physical" else k_max * self.type_dim type_head_out_dim = None if target == "physical" else k_max * self.type_dim
self._build_trunk_and_heads( self._build_trunk_and_heads(
@@ -386,11 +402,6 @@ class Stage2OneShot(StageModel):
type_head_cfg=type_head_cfg, type_head_cfg=type_head_cfg,
) )
def _cond_embed(self, cond_cont: torch.Tensor, cond_cat: torch.Tensor, stage1_out: torch.Tensor) -> torch.Tensor:
base = self.cond_enc(cond_cont, cond_cat)
ctx = self.context_adapter(stage1_out)
return self.fuse(torch.cat([base, ctx], dim=-1))
def forward( def forward(
self, self,
x_t: torch.Tensor, x_t: torch.Tensor,
@@ -702,11 +713,25 @@ class Stage2Autoregressive(StageModel):
return self.stop_head(c_emb.reshape(B * K, -1)).view(B, K) return self.stop_head(c_emb.reshape(B * K, -1)).view(B, K)
class CriticModel(nn.Module): class CriticModel(StageModel):
"""Generator-agnostic WGAN-GP critic body: a scalar realism score, for """Generator-agnostic WGAN-GP critic body: a scalar realism score, for
either stage (`stage="stage1"` mirrors v0.2 `Critic`; `stage="stage2"` either stage (`stage="stage1"` mirrors v0.2 `Critic`; `stage="stage2"`
mirrors v0.2 `SecondaryCritic`, adding the same context-fusion path as mirrors v0.2 `SecondaryCritic`, adding the same context-fusion path as
`Stage2OneShot`). Used only when that stage's `generator == "wgan"`.""" `Stage2OneShot`, via `StageModel._build_context_fusion`/`_cond_embed`).
Used only when that stage's `generator == "wgan"`.
Subclasses `StageModel` for the `cond_enc` construction and (stage 2)
context-fusion scaffolding only its trunk is built directly via
`build_trunk` (output width 1) rather than through
`_build_trunk_and_heads`, since that helper is shaped around a
generator's `Objective`/time-embedding/flow-matching concerns
(`forward`'s `(x_t, cond) -> vector` shape) that don't apply to a critic's
`(x, cond) -> scalar` (gitea #57). `generator="wgan"` is passed to the
base purely because that's factually when a critic exists; nothing here
ever calls `_build_trunk_and_heads`, so no head/time-embedding machinery
is built from it. Never routed (MoE) that's a separate, unrequested
axis of scope; see gitea #57's proposal, which covers only the trunk/
block registries."""
def __init__( def __init__(
self, self,
@@ -722,22 +747,26 @@ class CriticModel(nn.Module):
stage: str = "stage1", stage: str = "stage1",
context_dim: int = 64, context_dim: int = 64,
context_in_dim: int = X_DIM, context_in_dim: int = X_DIM,
trunk_type: str = "resmlp",
block_conditioning: str = "add",
) -> None: ) -> None:
super().__init__() super().__init__(
pdg_vocab,
mat_vocab,
particle_cfg,
material_cfg,
cond_out_dim=cond_out_dim,
generator="wgan",
noise_dim=0,
)
if stage not in ("stage1", "stage2"): if stage not in ("stage1", "stage2"):
raise ValueError(f"stage must be 'stage1' or 'stage2', got {stage!r}") raise ValueError(f"stage must be 'stage1' or 'stage2', got {stage!r}")
self.stage = stage self.stage = stage
self.cond_enc = ConditionEncoder(pdg_vocab, mat_vocab, particle_cfg, material_cfg, out_dim=cond_out_dim)
if stage == "stage2": if stage == "stage2":
self.context_adapter = ContextAdapter(context_in_dim, context_dim) self._build_context_fusion(context_in_dim, context_dim, cond_out_dim)
self.fuse = nn.Sequential( self.trunk = build_trunk(
nn.Linear(cond_out_dim + context_dim, cond_out_dim), None, trunk_type, in_dim, 1, hidden_dim, n_res_blocks, cond_out_dim, dropout, block_conditioning
nn.SiLU(), )
)
self.input_proj = nn.Linear(in_dim, hidden_dim)
self.blocks = nn.ModuleList([ResBlock(hidden_dim, cond_out_dim, dropout=dropout) for _ in range(n_res_blocks)])
self.out_norm = nn.LayerNorm(hidden_dim)
self.out_proj = nn.Linear(hidden_dim, 1)
def forward( def forward(
self, self,
@@ -746,13 +775,9 @@ class CriticModel(nn.Module):
cond_cat: torch.Tensor, cond_cat: torch.Tensor,
stage1_out: torch.Tensor | None = None, stage1_out: torch.Tensor | None = None,
) -> torch.Tensor: ) -> torch.Tensor:
base = self.cond_enc(cond_cont, cond_cat)
if self.stage == "stage2": if self.stage == "stage2":
ctx = self.context_adapter(stage1_out) assert stage1_out is not None, "stage='stage2' CriticModel requires stage1_out"
cond = self.fuse(torch.cat([base, ctx], dim=-1)) cond = self._cond_embed(cond_cont, cond_cat, stage1_out)
else: else:
cond = base cond = self.cond_enc(cond_cont, cond_cat)
h = self.input_proj(x) return self.trunk(x, cond, cond_cont, cond_cat).squeeze(-1)
for block in self.blocks:
h = block(h, cond)
return self.out_proj(self.out_norm(h)).squeeze(-1)
+3 -3
View File
@@ -28,6 +28,7 @@ import polars as pl
from giant.analysis.catalog import catalog_ids, get_spec from giant.analysis.catalog import catalog_ids, get_spec
from giant.analysis.condor import compute_reduced from giant.analysis.condor import compute_reduced
from giant.analysis.context import build_context from giant.analysis.context import build_context
from giant.analysis.sources import RolloutSpec
# Row counts (per side) to benchmark at. Kept in local memory/CPU range so the # Row counts (per side) to benchmark at. Kept in local memory/CPU range so the
# whole sweep finishes in about a minute; the fit is linear so it extrapolates # whole sweep finishes in about a minute; the fit is linear so it extrapolates
@@ -165,11 +166,10 @@ def _time(spec_id: str, rollout: Path, reference: Path, shared: Path, out: Path)
t0 = time.perf_counter() t0 = time.perf_counter()
compute_reduced( compute_reduced(
spec_id, spec_id,
rollout, [{"name": "rollout", "path": str(rollout)}],
reference, reference,
shared, shared,
out, out,
checkpoint=None,
chunk_index=0, chunk_index=0,
n_chunks=1, n_chunks=1,
) )
@@ -191,7 +191,7 @@ def main() -> None:
shared = tmp_path / f"shared_{n_side}.json" shared = tmp_path / f"shared_{n_side}.json"
ctx = build_context( ctx = build_context(
rollout, [RolloutSpec(name="rollout", source=rollout)],
reference, reference,
n_energy_bins=4, n_energy_bins=4,
n_marginal_bins=50, n_marginal_bins=50,
+356
View File
@@ -0,0 +1,356 @@
"""Training-progress plots from `<run_dir>/metrics.csv` (gitea #75).
`MetricsCollector` (`giant.training.metrics`) writes one row per epoch with a
column set that varies by run flow/ddpm vs wgan, routed vs not (see the
`MetricSpec` declarations in `giant.training.trainers`). This module reads
that header dynamically rather than hardcoding a column list, buckets columns
by the fixed naming convention `MetricsCollector` itself documents
(`<stage>/train/<key>`, `<stage>/val/<key>`, `<stage>/router/<key>`,
`<stage>/<key>` for point-in-time values, and an unprefixed run-level tail
see `giant.training.metrics`'s module docstring), and renders one PDF per
applicable figure with the same `plotstyle` conventions
`giant.analysis.render` uses, for visual consistency with the
rollout-vs-reference plots.
Unlike `giant.analysis`, there is no reduce/chunk/condor split here the CSV
is tiny and this always runs as one local pass but the CLI entry point
still lives under `giant analyze` (`analyze metrics`) as the shared home for
plotstyle-rendered diagnostics, and shares its `analysis_runs/` output
convention (see `derive_metrics_dir`) so training-progress plots don't get
written into the training run directory itself.
"""
from __future__ import annotations
import csv
import math
from dataclasses import dataclass
from pathlib import Path
# Stage names are always exactly these two — hardcoded in
# `giant.training.trainers.build_stage_trainers` — so a column belongs to a
# stage iff it's prefixed by one of these, and everything else (bar `epoch`)
# is run-level. This is what makes dynamic header parsing tractable without
# needing to know the per-run metric keys themselves.
_STAGE_NAMES = ("stage1", "stage2")
_ACC_KEYS = {"nsec_acc", "stop_acc", "type_acc"}
_WGAN_BALANCE_KEYS = {"d_loss", "g_loss", "wasserstein", "gp_loss"}
_ROUTER_KEYS = ("entropy", "util_min", "util_max", "util_std")
@dataclass
class MetricsTable:
"""`<run_dir>/metrics.csv`, parsed with no hardcoded column list."""
epochs: list[int]
columns: dict[str, list[float]]
@classmethod
def load(cls, path: str | Path) -> "MetricsTable":
with open(path, newline="") as f:
rows = list(csv.DictReader(f))
epochs = [int(float(r["epoch"])) for r in rows]
fieldnames = rows[0].keys() if rows else []
columns = {name: [float(r[name]) for r in rows] for name in fieldnames if name != "epoch"}
return cls(epochs=epochs, columns=columns)
def best_epochs(self) -> list[int]:
is_best = self.columns.get("is_best")
if not is_best:
return []
return [epoch for epoch, flag in zip(self.epochs, is_best) if flag]
# --- column classification --------------------------------------------------
def _stages(columns: dict) -> list[str]:
return [s for s in _STAGE_NAMES if any(name.startswith(f"{s}/") for name in columns)]
def _split(columns: dict, stage: str, split: str) -> dict[str, str]:
prefix = f"{stage}/{split}/"
return {name[len(prefix) :]: name for name in columns if name.startswith(prefix)}
def _point_in_time(columns: dict, stage: str) -> dict[str, str]:
prefix = f"{stage}/"
out = {}
for name in columns:
if not name.startswith(prefix):
continue
rest = name[len(prefix) :]
head = rest.split("/", 1)[0]
if head not in ("train", "val", "router"):
out[rest] = name
return out
def _router(columns: dict, stage: str) -> dict[str, str]:
prefix = f"{stage}/router/"
return {name[len(prefix) :]: name for name in columns if name.startswith(prefix)}
def _run_level(columns: dict) -> dict[str, str]:
known_prefixes = tuple(f"{s}/" for s in _STAGE_NAMES)
return {name: name for name in columns if not name.startswith(known_prefixes)}
def _loss_keys(train: dict[str, str], val: dict[str, str]) -> list[str]:
keys = {k for k in train if k not in _ACC_KEYS and k not in _WGAN_BALANCE_KEYS and k != "grad_norm"}
keys |= {k for k in val if k not in _ACC_KEYS and k not in _WGAN_BALANCE_KEYS and k != "grad_norm"}
return sorted(keys)
# --- output location ---------------------------------------------------------
def derive_metrics_dir(
run_dir: str | Path,
out_dir: str | Path | None = None,
default_base: str | Path | None = None,
) -> Path:
"""Plots output directory.
Precedence: an explicit `out_dir` always wins. Otherwise
`default_base / f"metrics_{run_dir.name}"` (the CLI passes the repo's
gitignored `analysis_runs/`, matching `giant.analysis.condor.derive_run_dir`'s
convention) training-progress plots live alongside rollout-vs-reference
analysis runs, not inside the training run directory itself.
"""
if out_dir is not None:
return Path(out_dir)
base = Path(default_base) if default_base is not None else Path.cwd() / "analysis_runs"
return base / f"metrics_{Path(run_dir).name}"
# --- figures ------------------------------------------------------------------
def _mark_best(ax, table: MetricsTable) -> None:
for epoch in table.best_epochs():
ax.axvline(epoch, color="grey", linestyle="--", linewidth=0.8, alpha=0.7)
def _overview_figure(table: MetricsTable):
import plotstyle as ps
run_level = _run_level(table.columns)
if "val/loss" not in run_level:
return None
fig, ax = ps.new_figure("thesis-single", title="training overview")
ax.plot(table.epochs, table.columns["val/loss"], label="val/loss")
if "val/marginal_kl" in run_level:
kl = table.columns["val/marginal_kl"]
if any(math.isfinite(v) for v in kl):
ax.plot(table.epochs, kl, label="val/marginal_kl")
_mark_best(ax, table)
best = table.best_epochs()
if best:
idx = table.epochs.index(best[-1])
ax.annotate(
f"best: epoch {best[-1]}\nval/loss={table.columns['val/loss'][idx]:.4g}",
xy=(best[-1], table.columns["val/loss"][idx]),
xytext=(0.98, 0.95),
textcoords="axes fraction",
ha="right",
va="top",
fontsize=8,
)
ax.set_xlabel("epoch")
ax.set_ylabel("loss")
ps.style_legend(ax, title="series")
return fig
def _loss_figure(table: MetricsTable, stage: str):
import plotstyle as ps
train = _split(table.columns, stage, "train")
val = _split(table.columns, stage, "val")
keys = _loss_keys(train, val)
if not keys:
return None
n = len(keys)
ncols = min(3, n)
nrows = (n + ncols - 1) // ncols
fig, axes = ps.new_figure(
"slide-16x9",
title=f"{stage} loss",
nrows=nrows,
ncols=ncols,
squeeze=False,
)
flat = axes.ravel()
for ax, key in zip(flat, keys):
if key in train:
ax.plot(table.epochs, table.columns[train[key]], label="train")
if key in val:
ax.plot(table.epochs, table.columns[val[key]], label="val")
ax.set_yscale("log")
ax.set_title(key, fontsize=8)
ax.set_xlabel("epoch")
for j in range(n, len(flat)):
flat[j].set_visible(False)
ps.style_legend(flat[0], title="series")
return fig
def _lr_figure(table: MetricsTable):
import plotstyle as ps
series: dict[str, str] = {}
for stage in _stages(table.columns):
for key, col in _point_in_time(table.columns, stage).items():
series[f"{stage}/{key}"] = col
if not series:
return None
fig, ax = ps.new_figure("thesis-single", title="learning rate schedule")
for label, col in series.items():
ax.plot(table.epochs, table.columns[col], label=label)
ax.set_xlabel("epoch")
ax.set_ylabel("learning rate")
ps.style_legend(ax, title="series")
return fig
def _accuracy_figure(table: MetricsTable, stage: str):
import plotstyle as ps
train = _split(table.columns, stage, "train")
val = _split(table.columns, stage, "val")
keys = sorted((set(train) | set(val)) & _ACC_KEYS)
if not keys:
return None
n = len(keys)
fig, axes = ps.new_figure("slide-16x9", title=f"{stage} accuracy", nrows=1, ncols=n, squeeze=False)
flat = axes.ravel()
for ax, key in zip(flat, keys):
if key in train:
ax.plot(table.epochs, table.columns[train[key]], label="train")
if key in val:
ax.plot(table.epochs, table.columns[val[key]], label="val")
ax.set_title(key, fontsize=8)
ax.set_xlabel("epoch")
ax.set_ylim(0, 1)
ps.style_legend(flat[0], title="series")
return fig
def _grad_norm_figure(table: MetricsTable):
import plotstyle as ps
run_level = _run_level(table.columns)
if "grad_norm" not in run_level:
return None
fig, ax = ps.new_figure("thesis-single", title="gradient norm")
ax.plot(table.epochs, table.columns["grad_norm"], label="grad_norm")
for stage in _stages(table.columns):
train = _split(table.columns, stage, "train")
for key in ("grad_norm_d", "grad_norm_g", "grad_norm_type_slice", "grad_norm_cont_slice"):
if key in train:
ax.plot(table.epochs, table.columns[train[key]], label=f"{stage}/{key}")
ax.set_yscale("log")
ax.set_xlabel("epoch")
ax.set_ylabel("grad norm")
ps.style_legend(ax, title="series")
return fig
def _router_figure(table: MetricsTable, stage: str):
import plotstyle as ps
router = _router(table.columns, stage)
if "entropy" not in router:
return None
fig, ax = ps.new_figure("thesis-single", title=f"{stage} router health")
ax.plot(table.epochs, table.columns[router["entropy"]], label="entropy", color="black")
ax.set_xlabel("epoch")
ax.set_ylabel("entropy [bits]")
ax2 = ax.twinx()
for key in ("util_min", "util_max", "util_std"):
if key in router:
ax2.plot(table.epochs, table.columns[router[key]], label=key, linestyle="--")
ax2.set_ylabel("expert utilization")
ax2.set_ylim(0, 1)
lines1, labels1 = ax.get_legend_handles_labels()
lines2, labels2 = ax2.get_legend_handles_labels()
ax.legend(lines1 + lines2, labels1 + labels2, loc="upper right", frameon=False, fontsize=7)
return fig
def _wgan_balance_figure(table: MetricsTable, stage: str):
import plotstyle as ps
train = _split(table.columns, stage, "train")
keys = [k for k in _WGAN_BALANCE_KEYS if k in train]
if not keys:
return None
fig, ax = ps.new_figure("thesis-single", title=f"{stage} WGAN critic/generator balance")
for key in sorted(keys):
ax.plot(table.epochs, table.columns[train[key]], label=key)
ax.set_xlabel("epoch")
ax.set_ylabel("value")
ps.style_legend(ax, title="series")
return fig
def _throughput_figure(table: MetricsTable):
import plotstyle as ps
run_level = _run_level(table.columns)
keys = [k for k in ("samples_per_sec", "gpu_mem_mb", "epoch_time_s") if k in run_level]
if not keys:
return None
fig, axes = ps.new_figure("slide-16x9", title="throughput / resources", nrows=1, ncols=len(keys), squeeze=False)
flat = axes.ravel()
for ax, key in zip(flat, keys):
ax.plot(table.epochs, table.columns[key])
_mark_best(ax, table)
ax.set_title(key, fontsize=8)
ax.set_xlabel("epoch")
return fig
# --- entry point ---------------------------------------------------------
def render_metrics(
run_dir: str | Path,
out_dir: str | Path | None = None,
default_base: str | Path | None = None,
) -> list[Path]:
"""`<run_dir>/metrics.csv` -> `<plots dir>/<name>.pdf`.
See `derive_metrics_dir` for how the plots directory is resolved.
"""
import matplotlib.pyplot as plt
import plotstyle as ps
ps.use()
table = MetricsTable.load(Path(run_dir) / "metrics.csv")
plots_dir = derive_metrics_dir(run_dir, out_dir, default_base)
plots_dir.mkdir(parents=True, exist_ok=True)
figures = [("overview", _overview_figure(table))]
for stage in _stages(table.columns):
figures.append((f"{stage}_loss", _loss_figure(table, stage)))
figures.append(("lr", _lr_figure(table)))
for stage in _stages(table.columns):
figures.append((f"{stage}_accuracy", _accuracy_figure(table, stage)))
figures.append(("grad_norm", _grad_norm_figure(table)))
for stage in _stages(table.columns):
figures.append((f"{stage}_router", _router_figure(table, stage)))
figures.append((f"{stage}_wgan_balance", _wgan_balance_figure(table, stage)))
figures.append(("throughput", _throughput_figure(table)))
paths: list[Path] = []
for name, fig in figures:
if fig is None:
continue
path = plots_dir / name
ps.savefig(fig, str(path), formats=("pdf",))
plt.close(fig)
paths.append(path.with_suffix(".pdf"))
return paths
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "giant" name = "giant"
version = "0.3.5" version = "0.3.11"
description = "Geant4 step-function surrogate via conditional flow matching" description = "Geant4 step-function surrogate via conditional flow matching"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
+14
View File
@@ -13,6 +13,7 @@ from giant.analysis.sources import (
open_side, open_side,
physical_steps, physical_steps,
secondaries, secondaries,
secondaries_by_step,
) )
from giant.data.loader import EVENT_ID_FILE_STRIDE from giant.data.loader import EVENT_ID_FILE_STRIDE
@@ -159,6 +160,19 @@ def test_secondaries_rollout_vs_reference_align():
assert t["pdg"].to_list() == [22, 22] assert t["pdg"].to_list() == [22, 22]
def test_secondaries_by_step_keys_each_secondary_to_its_emitting_step():
r = secondaries_by_step(_rollout_frame(), Side.rollout).collect()
assert r["pdg"].to_list() == [22]
# the rollout key is (event_id, parent_id, birth position) — the parent
# step's post_pos, copied verbatim onto the child's birth row.
assert r["step_key"][0] == {"event_id": 1, "parent_id": 0, "pre_x": 0.0, "pre_y": 0.0, "pre_z": 1.0}
t = secondaries_by_step(_reference_frame(), Side.reference).collect()
assert t["pdg"].to_list() == [22, 22]
# one row per emitting step; the empty-list step drops out entirely
assert [k["_row"] for k in t["step_key"]] == [0, 2]
def test_leakage_fraction(): def test_leakage_fraction():
frac = R.leakage_fraction(_rollout_frame()) frac = R.leakage_fraction(_rollout_frame())
# event 1: escaped pre_E=30, deposited=90 -> 30/120 = 0.25; event 2: 0 # event 1: escaped pre_E=30, deposited=90 -> 30/120 = 0.25; event 2: 0
+141 -26
View File
@@ -6,14 +6,29 @@ import numpy as np
import pytest import pytest
from giant.analysis import build_catalog, catalog_ids, get_spec from giant.analysis import build_catalog, catalog_ids, get_spec
from giant.analysis.catalog import Bundle, PlotSpec from giant.analysis.catalog import (
Bundle,
PlotSpec,
_containment_depths,
_ks_statistic,
)
from giant.analysis.context import Context, build_context from giant.analysis.context import Context, build_context
from giant.analysis.grouping import pdg_label
from giant.analysis.sources import RolloutSpec
from tests.test_analysis_reduce import _reference_frame, _rollout_frame from tests.test_analysis_reduce import _reference_frame, _rollout_frame
def _build_ctx() -> Context: def _build_ctx() -> Context:
r, t = _rollout_frame(), _reference_frame() r, t = _rollout_frame(), _reference_frame()
return build_context(r, t, n_energy_bins=2, n_marginal_bins=10, top_k_pdg=3, sample_rows=1000) return build_context(
[RolloutSpec("rollout", r)], t, n_energy_bins=2, n_marginal_bins=10, top_k_pdg=3, sample_rows=1000
)
def _two_rollout_specs() -> list[RolloutSpec]:
# Two distinct rollout sources so multi-series merging/finalize code is
# exercised even though the underlying frame is the same fixture.
return [RolloutSpec("flow", _rollout_frame()), RolloutSpec("wgan", _rollout_frame())]
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
@@ -21,9 +36,20 @@ def ctx() -> Context:
return _build_ctx() return _build_ctx()
@pytest.fixture(scope="module")
def two_ctx() -> Context:
t = _reference_frame()
return build_context(_two_rollout_specs(), t, n_energy_bins=2, n_marginal_bins=10, top_k_pdg=3, sample_rows=1000)
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
def bundle(ctx: Context) -> Bundle: def bundle(ctx: Context) -> Bundle:
return Bundle.open(_rollout_frame(), _reference_frame(), ctx) return Bundle.open([RolloutSpec("rollout", _rollout_frame())], _reference_frame(), ctx)
@pytest.fixture(scope="module")
def two_bundle(two_ctx: Context) -> Bundle:
return Bundle.open(_two_rollout_specs(), _reference_frame(), two_ctx)
def test_catalog_ids_unique_and_nonempty(): def test_catalog_ids_unique_and_nonempty():
@@ -53,41 +79,76 @@ def test_every_spec_computes_valid_reduced(bundle: Bundle):
"single_hist", "single_hist",
"router_gating", "router_gating",
"router_share", "router_share",
"router_specialization",
"heatmap",
"unavailable", "unavailable",
} }
assert r.title and r.xlabel assert r.title and r.xlabel
_validate_payload(r) _validate_payload(r, ["rollout"])
def _validate_payload(r) -> None: def test_every_spec_computes_valid_reduced_with_two_rollouts(two_bundle: Bundle):
for spec in build_catalog():
r = spec.finalize([spec.compute_partial(two_bundle)], two_bundle.ctx)
assert r.id == spec.id
_validate_payload(r, ["flow", "wgan"])
def _validate_payload(r, names: list[str]) -> None:
p = r.payload p = r.payload
if r.kind == "overlay_hist": if r.kind == "overlay_hist":
n = len(p["edges"]) - 1 n = len(p["edges"]) - 1
assert len(p["rollout"]) == n and len(p["reference"]) == n assert list(p["series"]) == names
for v in p["series"].values():
assert len(v) == n
assert len(p["reference"]) == n
elif r.kind == "single_hist": elif r.kind == "single_hist":
assert len(p["rollout"]) == len(p["edges"]) - 1 assert list(p["series"]) == names
for v in p["series"].values():
assert len(v) == len(p["edges"]) - 1
elif r.kind == "grouped_hist": elif r.kind == "grouped_hist":
n = len(p["edges"]) - 1 n = len(p["edges"]) - 1
assert p["groups"], "grouped hist must have at least one group" assert p["groups"], "grouped hist must have at least one group"
for g in p["groups"].values(): for g in p["groups"].values():
assert len(g["rollout"]) == n and len(g["reference"]) == n assert list(g["series"]) == names
for v in g["series"].values():
assert len(v) == n
assert len(g["reference"]) == n
elif r.kind == "profile": elif r.kind == "profile":
n = len(p["edges"]) - 1 n = len(p["edges"]) - 1
for k in ("rollout_mean", "rollout_std", "reference_mean", "reference_std"): assert list(p["series"]) == names
assert len(p[k]) == n for side in p["series"].values():
assert len(side["mean"]) == n and len(side["std"]) == n
assert len(p["reference"]["mean"]) == n and len(p["reference"]["std"]) == n
elif r.kind == "bar": elif r.kind == "bar":
assert len(p["labels"]) == len(p["rollout"]) == len(p["reference"]) assert list(p["series"]) == names
for v in p["series"].values():
assert len(p["labels"]) == len(v)
assert len(p["labels"]) == len(p["reference"])
elif r.kind == "unavailable": elif r.kind == "unavailable":
assert p["note"] assert p["note"]
elif r.kind == "router_gating": elif r.kind == "router_gating":
for side in ("rollout", "reference"): for entry in p["series"].values():
if side in p:
assert len(p[side]["centers"]) == len(p[side]["means"])
elif r.kind == "router_share":
for cat in p["categories"]:
for side in ("rollout", "reference"): for side in ("rollout", "reference"):
if side in p: if side in entry:
assert cat in p[side] assert len(entry[side]["centers"]) == len(entry[side]["means"])
elif r.kind == "router_share":
for entry in p["series"].values():
for cat in entry["categories"]:
for side in ("rollout", "reference"):
if side in entry:
assert cat in entry[side]
elif r.kind == "router_specialization":
for entry in p["series"].values():
for side in ("rollout", "reference"):
if side in entry:
assert len(entry[side]["centers"]) == len(entry[side]["score"])
elif r.kind == "heatmap":
assert list(p["series"]) == names
for mat in p["series"].values():
assert len(mat) == len(p["row_labels"])
for row in mat:
assert len(row) == len(p["col_labels"])
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -98,7 +159,11 @@ def _validate_payload(r) -> None:
# sec_count_per_species via pdg-keyed sums), concat-then-finalize with # sec_count_per_species via pdg-keyed sums), concat-then-finalize with
# data-dependent edges (event_total_edep), concat-then-mean/std (shower_ # data-dependent edges (event_total_edep), concat-then-mean/std (shower_
# longitudinal), concat-then-max-edge (leakage_fraction), pdg-keyed sum with a # longitudinal), concat-then-max-edge (leakage_fraction), pdg-keyed sum with a
# ratio (species_edep_share), and a chunkable=False passthrough (router_gating). # ratio (species_edep_share), a chunkable=False passthrough (router_gating),
# sum-mergeable-with-a-zero-fill-denominator (sec_count_per_step{,_by_species}),
# nested sum-merge into a scorecard (marginal_distance_summary), and
# concat-then-per-event-derived-quantity
# (shower_containment_depth_90, reusing the profile matrix's own merge shape).
_CHUNK_EQUIVALENCE_IDS = [ _CHUNK_EQUIVALENCE_IDS = [
"marginal_edep", "marginal_edep",
"species_edep_share", "species_edep_share",
@@ -106,7 +171,11 @@ _CHUNK_EQUIVALENCE_IDS = [
"shower_longitudinal", "shower_longitudinal",
"leakage_fraction", "leakage_fraction",
"sec_count_per_species", "sec_count_per_species",
"sec_count_per_step",
"sec_count_per_step_by_species",
"router_gating", "router_gating",
"marginal_distance_summary",
"shower_containment_depth_90",
] ]
@@ -128,21 +197,67 @@ def _assert_payload_close(a, b, path: str = "payload") -> None:
@pytest.mark.parametrize("spec_id", _CHUNK_EQUIVALENCE_IDS) @pytest.mark.parametrize("spec_id", _CHUNK_EQUIVALENCE_IDS)
def test_chunked_matches_unchunked(ctx: Context, spec_id: str): def test_chunked_matches_unchunked(two_ctx: Context, spec_id: str):
"""A plot computed over N event-disjoint chunks then merged must equal the """A plot computed over N event-disjoint chunks then merged must equal the
same plot computed in one unchunked pass the core chunking correctness same plot computed in one unchunked pass the core chunking correctness
guarantee (see the analysis-rollout-plots chunking plan).""" guarantee (see the analysis-rollout-plots chunking plan). Exercised with
two rollout series so the per-rollout merge path is covered too."""
spec: PlotSpec = get_spec(spec_id) spec: PlotSpec = get_spec(spec_id)
r, t = _rollout_frame(), _reference_frame() rollouts, t = _two_rollout_specs(), _reference_frame()
unchunked_bundle = Bundle.open(r, t, ctx) unchunked_bundle = Bundle.open(rollouts, t, two_ctx)
unchunked = spec.finalize([spec.compute_partial(unchunked_bundle)], ctx) unchunked = spec.finalize([spec.compute_partial(unchunked_bundle)], two_ctx)
# 4 chunks over only 2 distinct event_ids also exercises empty chunks. # 4 chunks over only 2 distinct event_ids also exercises empty chunks.
n_chunks = 4 if spec.chunkable else 1 n_chunks = 4 if spec.chunkable else 1
parts = [spec.compute_partial(Bundle.open(r, t, ctx, chunk=(k, n_chunks))) for k in range(n_chunks)] parts = [spec.compute_partial(Bundle.open(rollouts, t, two_ctx, chunk=(k, n_chunks))) for k in range(n_chunks)]
chunked = spec.finalize(parts, ctx) chunked = spec.finalize(parts, two_ctx)
assert chunked.id == unchunked.id assert chunked.id == unchunked.id
assert chunked.kind == unchunked.kind assert chunked.kind == unchunked.kind
_assert_payload_close(unchunked.payload, chunked.payload) _assert_payload_close(unchunked.payload, chunked.payload)
# ---------------------------------------------------------------------------
# new (gitea #76) reductions: KS distance and containment depth
# ---------------------------------------------------------------------------
def test_ks_statistic():
assert _ks_statistic([10, 10], [10, 10]) == 0.0 # identical shape -> 0
assert _ks_statistic([10, 0], [0, 10]) == 1.0 # fully disjoint -> 1
assert _ks_statistic([0, 0], [0, 0]) != _ks_statistic([0, 0], [0, 0]) # nan (no data either side)
assert _ks_statistic([10, 0], [0, 0]) == 1.0 # one side empty, other isn't -> maximal mismatch
def test_containment_depths_simple_ramp():
# one event, edep concentrated in the first bin -> 90%/95% containment
# depth is the first bin's right edge; a zero-energy event is dropped.
mat = np.array([[9.0, 1.0, 0.0], [0.0, 0.0, 0.0]])
edges = np.array([0.0, 1.0, 2.0, 3.0])
depths = _containment_depths(mat, edges, 0.90)
assert depths.tolist() == [1.0]
def test_sec_count_per_step_counts_empty_steps(bundle):
spec = get_spec("sec_count_per_step")
r = spec.finalize([spec.compute_partial(bundle)], bundle.ctx)
# reference: 3 steps, two of which emit exactly one secondary
assert r.payload["reference"][:2] == [1, 2]
# rollout: 4 physical steps, one of which emits a single secondary
assert r.payload["series"]["rollout"][:2] == [3, 1]
assert sum(r.payload["reference"]) == 3
def test_sec_count_per_step_by_species_zero_row_is_per_species(bundle):
spec = get_spec("sec_count_per_step_by_species")
r = spec.finalize([spec.compute_partial(bundle)], bundle.ctx)
cols = r.payload["col_labels"]
ref = r.payload["reference"]
g = cols.index(pdg_label(22))
# two reference steps emit one photon each; the third emits none
assert [row[g] for row in ref][:2] == [1, 2]
# every other species column is "no such secondary" on all 3 steps
for j, _ in enumerate(cols):
if j != g:
assert ref[0][j] == 3 and sum(row[j] for row in ref[1:]) == 0
+143 -30
View File
@@ -1,4 +1,4 @@
"""Tests for the rollout-YAML → run-directory flow, compute, and submit.""" """Tests for the rollout-YAML(s) → run-directory flow, compute, and submit."""
from __future__ import annotations from __future__ import annotations
@@ -17,6 +17,7 @@ from giant.analysis import (
compute_reduced, compute_reduced,
derive_run_dir, derive_run_dir,
load_rollout_yaml, load_rollout_yaml,
load_rollout_yamls,
merge_one, merge_one,
prep, prep,
write_submit, write_submit,
@@ -28,13 +29,17 @@ from giant.constants import PREDICT_COORD_METADATA_KEY, ROLLOUT_COORD_VALUE
from tests.test_analysis_reduce import _reference_frame, _rollout_frame from tests.test_analysis_reduce import _reference_frame, _rollout_frame
def _write_rollout(path: Path) -> None:
tbl = _rollout_frame().collect().to_arrow()
tbl = tbl.replace_schema_metadata({PREDICT_COORD_METADATA_KEY: ROLLOUT_COORD_VALUE})
pq.write_table(tbl, path)
def _write_inputs(tmp_path: Path) -> Path: def _write_inputs(tmp_path: Path) -> Path:
"""Materialize rollout+reference parquet and a rollout YAML; return the YAML path.""" """Materialize rollout+reference parquet and a rollout YAML; return the YAML path."""
rollout = tmp_path / "rollout.parquet" rollout = tmp_path / "rollout.parquet"
reference = tmp_path / "reference.parquet" reference = tmp_path / "reference.parquet"
tbl = _rollout_frame().collect().to_arrow() _write_rollout(rollout)
tbl = tbl.replace_schema_metadata({PREDICT_COORD_METADATA_KEY: ROLLOUT_COORD_VALUE})
pq.write_table(tbl, rollout)
_reference_frame().collect().write_parquet(reference) _reference_frame().collect().write_parquet(reference)
yaml_path = tmp_path / "run.yaml" yaml_path = tmp_path / "run.yaml"
@@ -54,6 +59,33 @@ def _write_inputs(tmp_path: Path) -> Path:
return yaml_path return yaml_path
def _write_two_inputs(tmp_path: Path) -> tuple[Path, Path]:
"""Two rollout YAMLs (distinct output files) sharing one reference file."""
reference = tmp_path / "reference.parquet"
_reference_frame().collect().write_parquet(reference)
paths = []
for tag, pred_id in (("a", "aaaa1111ef"), ("b", "bbbb2222ef")):
rollout = tmp_path / f"rollout_{tag}.parquet"
_write_rollout(rollout)
yaml_path = tmp_path / f"run_{tag}.yaml"
yaml_path.write_text(
yaml.safe_dump(
{
"prediction_id": pred_id,
"output": str(rollout),
"dataset": str(reference),
"checkpoint": f"/ckpt/{tag}.pt",
"kind": "rollout",
"energy_cutoff": 0.1,
"steps": 10,
}
)
)
paths.append(yaml_path)
return paths[0], paths[1]
def _fake_venv(repo_dir: Path) -> None: def _fake_venv(repo_dir: Path) -> None:
"""Stand in for a `uv sync`'d venv: write_submit checks `.venv/bin/giant` exists.""" """Stand in for a `uv sync`'d venv: write_submit checks `.venv/bin/giant` exists."""
giant = repo_dir / ".venv" / "bin" / "giant" giant = repo_dir / ".venv" / "bin" / "giant"
@@ -62,12 +94,13 @@ def _fake_venv(repo_dir: Path) -> None:
giant.chmod(0o755) giant.chmod(0o755)
def _prep(rollout_yaml: Path, run_dir: str | Path | None = None, chunks: int = 1) -> Path: def _prep(rollout_yamls, run_dir: str | Path | None = None, chunks: int = 1, labels=None) -> Path:
"""``prep`` with small test-sized context bins/sampling.""" """``prep`` with small test-sized context bins/sampling."""
return prep( return prep(
rollout_yaml, rollout_yamls,
run_dir, run_dir,
n_chunks=chunks, n_chunks=chunks,
labels=labels,
n_energy_bins=2, n_energy_bins=2,
n_marginal_bins=8, n_marginal_bins=8,
top_k_pdg=3, top_k_pdg=3,
@@ -82,39 +115,108 @@ def test_load_rollout_yaml_requires_paths(tmp_path: Path):
load_rollout_yaml(bad) load_rollout_yaml(bad)
def test_load_rollout_yamls_single_defaults_to_rollout_name(tmp_path: Path):
yaml_path = _write_inputs(tmp_path)
loaded, reference = load_rollout_yamls([yaml_path])
assert [lr.name for lr in loaded] == ["rollout"]
assert reference.endswith("reference.parquet")
def test_load_rollout_yamls_multi_defaults_to_stem(tmp_path: Path):
a, b = _write_two_inputs(tmp_path)
loaded, _ = load_rollout_yamls([a, b])
assert [lr.name for lr in loaded] == ["run_a", "run_b"]
def test_load_rollout_yamls_explicit_labels(tmp_path: Path):
a, b = _write_two_inputs(tmp_path)
loaded, _ = load_rollout_yamls([a, b], labels=["flow", "wgan"])
assert [lr.name for lr in loaded] == ["flow", "wgan"]
def test_load_rollout_yamls_label_count_mismatch(tmp_path: Path):
a, b = _write_two_inputs(tmp_path)
with pytest.raises(ValueError, match="--label"):
load_rollout_yamls([a, b], labels=["only-one"])
def test_load_rollout_yamls_rejects_duplicate_names(tmp_path: Path):
a, b = _write_two_inputs(tmp_path)
with pytest.raises(ValueError, match="collide"):
load_rollout_yamls([a, b], labels=["same", "same"])
def test_load_rollout_yamls_rejects_mismatched_reference(tmp_path: Path):
a, _ = _write_two_inputs(tmp_path)
other_ref = tmp_path / "other_reference.parquet"
_reference_frame().collect().write_parquet(other_ref)
c = tmp_path / "run_c.yaml"
c.write_text(
yaml.safe_dump(
{"prediction_id": "cccc3333ef", "output": str(tmp_path / "rollout_c.parquet"), "dataset": str(other_ref)}
)
)
_write_rollout(tmp_path / "rollout_c.parquet")
with pytest.raises(ValueError, match="same reference"):
load_rollout_yamls([a, c])
def test_derive_run_dir_next_to_rollout(): def test_derive_run_dir_next_to_rollout():
y = {"output": "/data/roll.parquet", "prediction_id": "abcd1234ef", "dataset": "d"} y = {"output": "/data/roll.parquet", "prediction_id": "abcd1234ef", "dataset": "d"}
assert derive_run_dir(y) == Path("/data/analysis_abcd1234") assert derive_run_dir([y]) == Path("/data/analysis_abcd1234")
assert derive_run_dir(y, "/somewhere") == Path("/somewhere") assert derive_run_dir([y], "/somewhere") == Path("/somewhere")
def test_derive_run_dir_default_base(): def test_derive_run_dir_default_base():
y = {"output": "/data/roll.parquet", "prediction_id": "abcd1234ef", "dataset": "d"} y = {"output": "/data/roll.parquet", "prediction_id": "abcd1234ef", "dataset": "d"}
assert derive_run_dir(y, default_base="/work/lbogner/giant2/analysis_runs") == Path( assert derive_run_dir([y], default_base="/work/lbogner/giant2/analysis_runs") == Path(
"/work/lbogner/giant2/analysis_runs/analysis_abcd1234" "/work/lbogner/giant2/analysis_runs/analysis_abcd1234"
) )
# an explicit run_dir still wins over default_base # an explicit run_dir still wins over default_base
assert derive_run_dir(y, "/somewhere", default_base="/other") == Path("/somewhere") assert derive_run_dir([y], "/somewhere", default_base="/other") == Path("/somewhere")
def test_derive_run_dir_multi_rollout_joins_tags():
ys = [{"output": f"/data/roll_{i}.parquet", "prediction_id": f"tag{i}xxxx", "dataset": "d"} for i in range(2)]
assert derive_run_dir(ys, default_base="/base") == Path("/base/analysis_tag0xxxx-tag1xxxx")
def test_derive_run_dir_many_rollouts_truncates_with_plus_count():
ys = [{"output": f"/data/roll_{i}.parquet", "prediction_id": f"tag{i}xxxx", "dataset": "d"} for i in range(5)]
run_dir = derive_run_dir(ys, default_base="/base")
assert run_dir == Path("/base/analysis_tag0xxxx-tag1xxxx-tag2xxxx-plus2")
def test_prep_lays_out_run_dir(tmp_path: Path): def test_prep_lays_out_run_dir(tmp_path: Path):
yaml_path = _write_inputs(tmp_path) yaml_path = _write_inputs(tmp_path)
run_dir = _prep(yaml_path) run_dir = _prep([yaml_path])
assert run_dir == tmp_path / "analysis_abcd1234" assert run_dir == tmp_path / "analysis_abcd1234"
assert (run_dir / "shared.json").exists() assert (run_dir / "shared.json").exists()
ctx = Context.load(run_dir / "shared.json") ctx = Context.load(run_dir / "shared.json")
assert set(ctx.var_ranges) == {"step_length", "edep", "delta_e", "post_E"} assert set(ctx.var_ranges) == {"step_length", "edep", "delta_e", "post_E"}
meta = RunMeta.load(run_dir / "run_meta.json") meta = RunMeta.load(run_dir / "run_meta.json")
assert meta.reference.endswith("reference.parquet") assert meta.reference.endswith("reference.parquet")
assert meta.plot_meta["checkpoint"] == "/ckpt/best.pt" assert [ro["name"] for ro in meta.rollouts] == ["rollout"]
assert meta.rollouts[0]["plot_meta"]["checkpoint"] == "/ckpt/best.pt"
assert "best.pt" in meta.title assert "best.pt" in meta.title
assert meta.n_chunks == 1 assert meta.n_chunks == 1
assert meta.rows_per_chunk == [meta.total_rows] # single chunk holds everything assert meta.rows_per_chunk == [meta.total_rows] # single chunk holds everything
assert meta.total_rows == 8 # 5 rollout rows + 3 reference rows assert meta.total_rows == 8 # 5 rollout rows + 3 reference rows
def test_prep_multi_rollout_lays_out_run_dir(tmp_path: Path):
a, b = _write_two_inputs(tmp_path)
run_dir = _prep([a, b], labels=["flow", "wgan"])
meta = RunMeta.load(run_dir / "run_meta.json")
assert [ro["name"] for ro in meta.rollouts] == ["flow", "wgan"]
assert meta.rollouts[0]["plot_meta"]["checkpoint"] == "/ckpt/a.pt"
assert meta.rollouts[1]["plot_meta"]["checkpoint"] == "/ckpt/b.pt"
# 5 rows from each rollout + 3 from the shared reference
assert meta.total_rows == 13
def test_prep_splits_rows_per_chunk(tmp_path: Path): def test_prep_splits_rows_per_chunk(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path), chunks=2) run_dir = _prep([_write_inputs(tmp_path)], chunks=2)
meta = RunMeta.load(run_dir / "run_meta.json") meta = RunMeta.load(run_dir / "run_meta.json")
assert len(meta.rows_per_chunk) == 2 assert len(meta.rows_per_chunk) == 2
assert sum(meta.rows_per_chunk) == meta.total_rows == 8 assert sum(meta.rows_per_chunk) == meta.total_rows == 8
@@ -125,7 +227,7 @@ def test_reprep_clears_stale_partials_from_a_different_chunk_count(tmp_path: Pat
partials on disk for merge_one to silently merge against the new partials on disk for merge_one to silently merge against the new
context (they'd be keyed/sized for the old n_chunks).""" context (they'd be keyed/sized for the old n_chunks)."""
yaml_path = _write_inputs(tmp_path) yaml_path = _write_inputs(tmp_path)
run_dir = _prep(yaml_path, chunks=2) run_dir = _prep([yaml_path], chunks=2)
compute_one("marginal_edep", run_dir, chunk_index=0) compute_one("marginal_edep", run_dir, chunk_index=0)
compute_one("marginal_edep", run_dir, chunk_index=1) compute_one("marginal_edep", run_dir, chunk_index=1)
stale = run_dir / "reduced_partial" / "marginal_edep__0.json" stale = run_dir / "reduced_partial" / "marginal_edep__0.json"
@@ -133,7 +235,7 @@ def test_reprep_clears_stale_partials_from_a_different_chunk_count(tmp_path: Pat
(run_dir / "reduced").mkdir(exist_ok=True) (run_dir / "reduced").mkdir(exist_ok=True)
(run_dir / "reduced" / "marginal_edep.json").write_text("{}") (run_dir / "reduced" / "marginal_edep.json").write_text("{}")
_prep(yaml_path, run_dir, chunks=1) _prep([yaml_path], run_dir, chunks=1)
assert not stale.exists() assert not stale.exists()
assert not (run_dir / "reduced" / "marginal_edep.json").exists() assert not (run_dir / "reduced" / "marginal_edep.json").exists()
@@ -141,20 +243,22 @@ def test_reprep_clears_stale_partials_from_a_different_chunk_count(tmp_path: Pat
def test_compute_one_from_run_dir(tmp_path: Path): def test_compute_one_from_run_dir(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path)) run_dir = _prep([_write_inputs(tmp_path)])
out = compute_one("marginal_edep", run_dir) out = compute_one("marginal_edep", run_dir)
assert out == run_dir / "reduced_partial" / "marginal_edep__0.json" assert out == run_dir / "reduced_partial" / "marginal_edep__0.json"
partial = Partial.load(out) partial = Partial.load(out)
assert partial.id == "marginal_edep" and partial.chunk == 0 assert partial.id == "marginal_edep" and partial.chunk == 0
assert "r" in partial.data and "t" in partial.data assert "r" in partial.data and "t" in partial.data
assert list(partial.data["r"]) == ["rollout"]
def test_compute_reduced_explicit_paths(tmp_path: Path): def test_compute_reduced_explicit_paths(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path)) run_dir = _prep([_write_inputs(tmp_path)])
meta = RunMeta.load(run_dir / "run_meta.json") meta = RunMeta.load(run_dir / "run_meta.json")
rollouts = [{"name": ro["name"], "path": ro["path"]} for ro in meta.rollouts]
out = compute_reduced( out = compute_reduced(
"marginal_step_length", "marginal_step_length",
meta.rollout, rollouts,
meta.reference, meta.reference,
run_dir / "shared.json", run_dir / "shared.json",
tmp_path / "r.json", tmp_path / "r.json",
@@ -163,17 +267,17 @@ def test_compute_reduced_explicit_paths(tmp_path: Path):
def test_merge_one_produces_reduced(tmp_path: Path): def test_merge_one_produces_reduced(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path)) run_dir = _prep([_write_inputs(tmp_path)])
compute_one("marginal_edep", run_dir) compute_one("marginal_edep", run_dir)
out = merge_one("marginal_edep", run_dir) out = merge_one("marginal_edep", run_dir)
assert out == run_dir / "reduced" / "marginal_edep.json" assert out == run_dir / "reduced" / "marginal_edep.json"
reduced = Reduced.load(out) reduced = Reduced.load(out)
assert reduced.id == "marginal_edep" assert reduced.id == "marginal_edep"
assert len(reduced.payload["rollout"]) == len(reduced.payload["edges"]) - 1 assert len(reduced.payload["series"]["rollout"]) == len(reduced.payload["edges"]) - 1
def test_merge_one_fails_loudly_on_missing_chunk(tmp_path: Path): def test_merge_one_fails_loudly_on_missing_chunk(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path), chunks=2) run_dir = _prep([_write_inputs(tmp_path)], chunks=2)
compute_one("marginal_edep", run_dir, chunk_index=0) # chunk 1 never computed compute_one("marginal_edep", run_dir, chunk_index=0) # chunk 1 never computed
with pytest.raises(FileNotFoundError, match="missing chunk"): with pytest.raises(FileNotFoundError, match="missing chunk"):
merge_one("marginal_edep", run_dir) merge_one("marginal_edep", run_dir)
@@ -182,11 +286,11 @@ def test_merge_one_fails_loudly_on_missing_chunk(tmp_path: Path):
def test_chunked_compute_and_merge_matches_unchunked(tmp_path: Path): def test_chunked_compute_and_merge_matches_unchunked(tmp_path: Path):
(tmp_path / "a").mkdir() (tmp_path / "a").mkdir()
(tmp_path / "b").mkdir() (tmp_path / "b").mkdir()
unchunked_dir = _prep(_write_inputs(tmp_path / "a")) unchunked_dir = _prep([_write_inputs(tmp_path / "a")])
compute_one("marginal_step_length", unchunked_dir) compute_one("marginal_step_length", unchunked_dir)
unchunked = Reduced.load(merge_one("marginal_step_length", unchunked_dir)) unchunked = Reduced.load(merge_one("marginal_step_length", unchunked_dir))
chunked_dir = _prep(_write_inputs(tmp_path / "b"), chunks=2) chunked_dir = _prep([_write_inputs(tmp_path / "b")], chunks=2)
for k in range(2): for k in range(2):
compute_one("marginal_step_length", chunked_dir, chunk_index=k) compute_one("marginal_step_length", chunked_dir, chunk_index=k)
chunked = Reduced.load(merge_one("marginal_step_length", chunked_dir)) chunked = Reduced.load(merge_one("marginal_step_length", chunked_dir))
@@ -194,14 +298,23 @@ def test_chunked_compute_and_merge_matches_unchunked(tmp_path: Path):
assert chunked.payload == unchunked.payload assert chunked.payload == unchunked.payload
def test_two_rollout_compute_and_merge_produces_both_series(tmp_path: Path):
a, b = _write_two_inputs(tmp_path)
run_dir = _prep([a, b], labels=["flow", "wgan"])
compute_one("marginal_edep", run_dir)
reduced = Reduced.load(merge_one("marginal_edep", run_dir))
assert list(reduced.payload["series"]) == ["flow", "wgan"]
assert "reference" in reduced.payload
def test_compute_reduced_rejects_out_of_range_chunk(tmp_path: Path): def test_compute_reduced_rejects_out_of_range_chunk(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path)) # n_chunks=1 (default) run_dir = _prep([_write_inputs(tmp_path)]) # n_chunks=1 (default)
with pytest.raises(ValueError, match="out of range"): with pytest.raises(ValueError, match="out of range"):
compute_one("marginal_edep", run_dir, chunk_index=1) compute_one("marginal_edep", run_dir, chunk_index=1)
def test_write_submit_description(tmp_path: Path): def test_write_submit_description(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path)) run_dir = _prep([_write_inputs(tmp_path)])
_fake_venv(tmp_path) _fake_venv(tmp_path)
cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path) cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path)
txt = write_submit(cfg).read_text() txt = write_submit(cfg).read_text()
@@ -223,7 +336,7 @@ def test_write_submit_description(tmp_path: Path):
def test_write_submit_requires_synced_venv(tmp_path: Path, monkeypatch: pytest.MonkeyPatch): def test_write_submit_requires_synced_venv(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
run_dir = _prep(_write_inputs(tmp_path)) run_dir = _prep([_write_inputs(tmp_path)])
cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path) cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path)
# No `giant` next to the (fake) active interpreter, so this falls through # No `giant` next to the (fake) active interpreter, so this falls through
# to repo_dir/.venv/bin/giant, which _write_inputs/_prep also didn't create. # to repo_dir/.venv/bin/giant, which _write_inputs/_prep also didn't create.
@@ -233,7 +346,7 @@ def test_write_submit_requires_synced_venv(tmp_path: Path, monkeypatch: pytest.M
def test_write_submit_remote_flag(tmp_path: Path): def test_write_submit_remote_flag(tmp_path: Path):
run_dir = _prep(_write_inputs(tmp_path)) run_dir = _prep([_write_inputs(tmp_path)])
_fake_venv(tmp_path) _fake_venv(tmp_path)
cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, remote=True) cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, remote=True)
txt = write_submit(cfg).read_text() txt = write_submit(cfg).read_text()
@@ -243,7 +356,7 @@ def test_write_submit_remote_flag(tmp_path: Path):
def test_write_submit_chunks_respect_chunkable(tmp_path: Path): def test_write_submit_chunks_respect_chunkable(tmp_path: Path):
assert get_spec("router_gating").chunkable is False assert get_spec("router_gating").chunkable is False
run_dir = _prep(_write_inputs(tmp_path), chunks=4) run_dir = _prep([_write_inputs(tmp_path)], chunks=4)
_fake_venv(tmp_path) _fake_venv(tmp_path)
cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, n_chunks=4) cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, n_chunks=4)
write_submit(cfg) write_submit(cfg)
@@ -260,7 +373,7 @@ def test_write_submit_rejects_n_chunks_mismatch_with_run_meta(tmp_path: Path):
with RunMeta.rows_per_chunk is sized to the prepped value, so a with RunMeta.rows_per_chunk is sized to the prepped value, so a
mismatch would otherwise surface as a confusing IndexError deep inside mismatch would otherwise surface as a confusing IndexError deep inside
_job_walltimes instead of a clear error here.""" _job_walltimes instead of a clear error here."""
run_dir = _prep(_write_inputs(tmp_path), chunks=2) run_dir = _prep([_write_inputs(tmp_path)], chunks=2)
_fake_venv(tmp_path) _fake_venv(tmp_path)
cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, n_chunks=4) cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, n_chunks=4)
with pytest.raises(ValueError, match="n_chunks"): with pytest.raises(ValueError, match="n_chunks"):
@@ -282,7 +395,7 @@ def test_write_submit_walltime_grows_with_chunk_rows(tmp_path: Path):
"""A chunked run's later job walltimes track that chunk's row count.""" """A chunked run's later job walltimes track that chunk's row count."""
from giant.analysis.runtime_estimate import estimate_runtime_s from giant.analysis.runtime_estimate import estimate_runtime_s
run_dir = _prep(_write_inputs(tmp_path), chunks=2) run_dir = _prep([_write_inputs(tmp_path)], chunks=2)
meta = RunMeta.load(run_dir / "run_meta.json") meta = RunMeta.load(run_dir / "run_meta.json")
_fake_venv(tmp_path) _fake_venv(tmp_path)
cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, n_chunks=2) cfg = SubmitConfig(run_dir=run_dir, accounting_group="cms", repo_dir=tmp_path, n_chunks=2)
+45 -11
View File
@@ -8,7 +8,10 @@ from giant.model.network import (
HISTORY_REGISTRY, HISTORY_REGISTRY,
AttentionHistory, AttentionHistory,
ConditionEncoder, ConditionEncoder,
CriticModel,
FilmResBlock,
HistoryEncoder, HistoryEncoder,
LinearTrunk,
MarkovHistory, MarkovHistory,
NoHistory, NoHistory,
SinusoidalEmbedding, SinusoidalEmbedding,
@@ -944,7 +947,7 @@ def test_build_critics_particle_type_n_classes_overrides_conditioning_emb_dim():
assert wider_critic is not None assert wider_critic is not None
# k_max=3 slots, each CONT_SLOT_DIM + n_classes wide under wgan folding — # k_max=3 slots, each CONT_SLOT_DIM + n_classes wide under wgan folding —
# widening n_classes alone (emb_dim stays 4) must widen the critic input. # widening n_classes alone (emb_dim stays 4) must widen the critic input.
assert wider_critic.input_proj.in_features > default_n_classes_critic.input_proj.in_features assert wider_critic.trunk.input_proj.in_features > default_n_classes_critic.trunk.input_proj.in_features
# ── build_models/build_critics: DEFAULT_CONFIG fallback drift (issues.md #1) ─ # ── build_models/build_critics: DEFAULT_CONFIG fallback drift (issues.md #1) ─
@@ -1021,12 +1024,12 @@ def test_build_critics_omitted_particle_type_matches_default_config():
cfg["stage2_model"]["generator"] = "wgan" cfg["stage2_model"]["generator"] = "wgan"
onehot_critic = build_critics(cfg)["stage2"] onehot_critic = build_critics(cfg)["stage2"]
assert onehot_critic is not None assert onehot_critic is not None
onehot_in_dim = onehot_critic.input_proj.in_features onehot_in_dim = onehot_critic.trunk.input_proj.in_features
cfg["stage2_model"]["particle_type"] = {"target": "physical"} cfg["stage2_model"]["particle_type"] = {"target": "physical"}
physical_critic = build_critics(cfg)["stage2"] physical_critic = build_critics(cfg)["stage2"]
assert physical_critic is not None assert physical_critic is not None
physical_in_dim = physical_critic.input_proj.in_features physical_in_dim = physical_critic.trunk.input_proj.in_features
# onehot's per-slot type width is emb_dim classes vs. physical's fixed # onehot's per-slot type width is emb_dim classes vs. physical's fixed
# (log-mass, charge) pair — different unless emb_dim happens to be 2, so # (log-mass, charge) pair — different unless emb_dim happens to be 2, so
@@ -1046,15 +1049,15 @@ def test_build_critics_stage1_critic_hidden_dim_and_n_res_blocks_override_genera
inherited = build_critics(cfg)["stage1"] inherited = build_critics(cfg)["stage1"]
assert inherited is not None assert inherited is not None
assert inherited.input_proj.out_features == 8 assert inherited.trunk.input_proj.out_features == 8
assert len(inherited.blocks) == 1 assert len(inherited.trunk.blocks) == 1
cfg["stage1_model"]["wgan"]["critic_hidden_dim"] = 16 cfg["stage1_model"]["wgan"]["critic_hidden_dim"] = 16
cfg["stage1_model"]["wgan"]["critic_n_res_blocks"] = 3 cfg["stage1_model"]["wgan"]["critic_n_res_blocks"] = 3
overridden = build_critics(cfg)["stage1"] overridden = build_critics(cfg)["stage1"]
assert overridden is not None assert overridden is not None
assert overridden.input_proj.out_features == 16 assert overridden.trunk.input_proj.out_features == 16
assert len(overridden.blocks) == 3 assert len(overridden.trunk.blocks) == 3
def test_build_critics_stage2_critic_hidden_dim_and_n_res_blocks_override_generator_size(): def test_build_critics_stage2_critic_hidden_dim_and_n_res_blocks_override_generator_size():
@@ -1065,15 +1068,15 @@ def test_build_critics_stage2_critic_hidden_dim_and_n_res_blocks_override_genera
inherited = build_critics(cfg)["stage2"] inherited = build_critics(cfg)["stage2"]
assert inherited is not None assert inherited is not None
assert inherited.input_proj.out_features == 8 assert inherited.trunk.input_proj.out_features == 8
assert len(inherited.blocks) == 1 assert len(inherited.trunk.blocks) == 1
cfg["stage2_model"]["wgan"]["critic_hidden_dim"] = 16 cfg["stage2_model"]["wgan"]["critic_hidden_dim"] = 16
cfg["stage2_model"]["wgan"]["critic_n_res_blocks"] = 3 cfg["stage2_model"]["wgan"]["critic_n_res_blocks"] = 3
overridden = build_critics(cfg)["stage2"] overridden = build_critics(cfg)["stage2"]
assert overridden is not None assert overridden is not None
assert overridden.input_proj.out_features == 16 assert overridden.trunk.input_proj.out_features == 16
assert len(overridden.blocks) == 3 assert len(overridden.trunk.blocks) == 3
# ── StageModel base (gitea #39): Stage1Model/Stage2OneShot/Stage2Autoregressive # ── StageModel base (gitea #39): Stage1Model/Stage2OneShot/Stage2Autoregressive
@@ -1234,3 +1237,34 @@ def test_stagemodel_time_emb_matches_objective_needs_time(cls, generator):
assert model.generator_kind == generator assert model.generator_kind == generator
assert model.noise_dim == 8 assert model.noise_dim == 8
assert (model.time_emb is not None) == build_objective(generator).needs_time assert (model.time_emb is not None) == build_objective(generator).needs_time
# ── CriticModel uses the trunk/block registries + StageModel base (gitea #57) ─
def test_critic_model_is_stagemodel_subclass():
assert issubclass(CriticModel, StageModel)
@pytest.mark.parametrize("stage", ["stage1", "stage2"])
def test_build_critics_threads_trunk_type_from_generator_config(stage):
cfg = _minimal_model_config(share_stages=False)
cfg["stage1_model"]["generator"] = "wgan"
cfg["stage2_model"]["generator"] = "wgan"
cfg[f"{stage}_model"]["trunk"] = {"type": "linear"}
critic = build_critics(cfg)[stage]
assert critic is not None
assert isinstance(critic.trunk, LinearTrunk)
@pytest.mark.parametrize("stage", ["stage1", "stage2"])
def test_build_critics_threads_block_conditioning_from_generator_config(stage):
cfg = _minimal_model_config(share_stages=False)
cfg["stage1_model"]["generator"] = "wgan"
cfg["stage2_model"]["generator"] = "wgan"
cfg[f"{stage}_model"]["trunk"] = {"block_conditioning": "film"}
critic = build_critics(cfg)[stage]
assert critic is not None
assert all(isinstance(block, FilmResBlock) for block in critic.trunk.blocks)
+165 -43
View File
@@ -25,41 +25,96 @@ def test_render_router_diagnostics_and_edge_cases(tmp_path: Path):
reduced = [ reduced = [
Reduced( Reduced(
"rg", "rg",
"router", "model",
"router_gating", "router_gating",
"Router gating", "Router gating",
"pre-step energy [MeV]", "pre-step energy [MeV]",
{ {
"n_experts": 2,
"log_x": True, "log_x": True,
"router_type": "energy", "series": {
"rollout": { "flow": {
"centers": [1.0, 10.0, 100.0], "n_experts": 2,
"means": [[0.6, 0.4], [0.5, 0.5], [0.4, 0.6]], "router_type": "energy",
}, "rollout": {
"reference": { "centers": [1.0, 10.0, 100.0],
"centers": [1.0, 10.0, 100.0], "means": [[0.6, 0.4], [0.5, 0.5], [0.4, 0.6]],
"means": [[0.55, 0.45], [0.5, 0.5], [0.45, 0.55]], },
"reference": {
"centers": [1.0, 10.0, 100.0],
"means": [[0.55, 0.45], [0.5, 0.5], [0.45, 0.55]],
},
},
"wgan": {
"n_experts": 2,
"router_type": "energy",
"rollout": {"centers": [1.0], "means": [[0.5, 0.5]]},
"reference": {"centers": [1.0], "means": [[0.5, 0.5]]},
},
}, },
}, },
), ),
Reduced( Reduced(
"rs", "rs",
"router", "model",
"router_share", "router_share",
"Router share", "Router share",
"species", "species",
{ {
"categories": ["e-", "gamma"], "series": {
"n_experts": 2, "flow": {
"router_type": "energy", "categories": ["e-", "gamma"],
"rollout": {"e-": [0.7, 0.3], "gamma": [0.2, 0.8]}, "n_experts": 2,
"reference": {"e-": [0.6, 0.4], "gamma": [0.3, 0.7]}, "router_type": "energy",
"rollout": {"e-": [0.7, 0.3], "gamma": [0.2, 0.8]},
"reference": {"e-": [0.6, 0.4], "gamma": [0.3, 0.7]},
},
},
},
),
Reduced(
"rp",
"model",
"router_share",
"Router share by process (reference-only)",
"process",
{
"series": {
"flow": {
"categories": ["compt", "phot"],
"n_experts": 2,
"router_type": "energy",
"reference": {"compt": [0.4, 0.6], "phot": [0.9, 0.1]},
},
},
},
),
Reduced(
"rz",
"model",
"router_specialization",
"Router specialization",
"pre-step energy [MeV]",
{
"log_x": True,
"series": {
"flow": {
"n_experts": 2,
"chance_level": 0.5,
"rollout": {"centers": [1.0, 10.0], "score": [0.6, 0.7]},
"reference": {"centers": [1.0, 10.0], "score": [0.55, 0.65]},
},
"wgan": {
"n_experts": 4,
"chance_level": 0.25,
"rollout": {"centers": [1.0, 10.0], "score": [0.3, 0.4]},
"reference": {"centers": [], "score": []},
},
},
}, },
), ),
Reduced( Reduced(
"ru", "ru",
"router", "model",
"unavailable", "unavailable",
"Router unavailable", "Router unavailable",
"x", "x",
@@ -73,7 +128,10 @@ def test_render_router_diagnostics_and_edge_cases(tmp_path: Path):
"x", "x",
{ {
"edges": [0, 1, 2], "edges": [0, 1, 2],
"groups": {lbl: {"rollout": [1, 2], "reference": [2, 1]} for lbl in ("a", "b", "c", "d")}, "groups": {
lbl: {"series": {"flow": [1, 2], "wgan": [2, 1]}, "reference": [2, 1]}
for lbl in ("a", "b", "c", "d")
},
"log_y": True, "log_y": True,
}, },
), ),
@@ -83,7 +141,22 @@ def test_render_router_diagnostics_and_edge_cases(tmp_path: Path):
"single_hist", "single_hist",
"Single (log-x)", "Single (log-x)",
"x", "x",
{"edges": [1, 10, 100], "rollout": [5, 1], "log_x": True, "log_y": True}, {"edges": [1, 10, 100], "series": {"flow": [5, 1], "wgan": [3, 2]}, "log_x": True, "log_y": True},
),
Reduced(
"hm",
"quality",
"heatmap",
"Distance summary (2 rollouts)",
"grouping axis",
{
"series": {"flow": [[0.1, 0.2], [0.3, 0.4]], "wgan": [[0.5, 0.6], [0.7, 0.8]]},
"row_labels": ["step_length", "edep"],
"col_labels": ["overall", "energy"],
"cbar_label": "KS statistic",
"vmin": 0.0,
"vmax": 1.0,
},
), ),
] ]
try: try:
@@ -117,7 +190,7 @@ def test_render_all_run_gallery_invokes_subprocess(tmp_path: Path, monkeypatch):
"single_hist", "single_hist",
"Single", "Single",
"x", "x",
{"edges": [0, 1, 2], "rollout": [5, 1]}, {"edges": [0, 1, 2], "series": {"rollout": [5, 1]}},
) )
] ]
for r in reduced: for r in reduced:
@@ -142,15 +215,14 @@ def test_render_run_glues_condor_run_meta_into_render_all(tmp_path: Path, monkey
merge_calls = [] merge_calls = []
monkeypatch.setattr(condor_mod, "merge_all", lambda rd: merge_calls.append(Path(rd))) monkeypatch.setattr(condor_mod, "merge_all", lambda rd: merge_calls.append(Path(rd)))
meta = condor_mod.RunMeta( meta = condor_mod.RunMeta(
rollout="rollout.parquet", rollouts=[{"name": "rollout", "path": "rollout.parquet", "plot_meta": {"checkpoint": "ckpt/best.pt"}}],
reference="reference.parquet", reference="reference.parquet",
run_dir=str(run_dir), run_dir=str(run_dir),
title="my-run", title="my-run",
plot_meta={"checkpoint": "ckpt/best.pt"},
) )
monkeypatch.setattr(condor_mod.RunMeta, "load", classmethod(lambda cls, p: meta)) monkeypatch.setattr(condor_mod.RunMeta, "load", classmethod(lambda cls, p: meta))
Reduced("s", "species", "single_hist", "Single", "x", {"edges": [0, 1], "rollout": [1]}).save( Reduced("s", "species", "single_hist", "Single", "x", {"edges": [0, 1], "series": {"rollout": [1]}}).save(
run_dir / "reduced" / "s.json" run_dir / "reduced" / "s.json"
) )
@@ -177,7 +249,7 @@ def test_render_one_of_each_kind(tmp_path: Path):
"x", "x",
{ {
"edges": [0, 1, 2, 3], "edges": [0, 1, 2, 3],
"rollout": [1, 2, 3], "series": {"flow": [1, 2, 3], "wgan": [2, 2, 2]},
"reference": [3, 2, 1], "reference": [3, 2, 1],
"log_y": False, "log_y": False,
}, },
@@ -190,7 +262,7 @@ def test_render_one_of_each_kind(tmp_path: Path):
"x", "x",
{ {
"edges": [0, 1, 2], "edges": [0, 1, 2],
"groups": {"a": {"rollout": [1, 2], "reference": [2, 1]}}, "groups": {"a": {"series": {"flow": [1, 2]}, "reference": [2, 1]}},
"log_y": False, "log_y": False,
}, },
), ),
@@ -202,10 +274,8 @@ def test_render_one_of_each_kind(tmp_path: Path):
"depth", "depth",
{ {
"edges": [0, 1, 2], "edges": [0, 1, 2],
"rollout_mean": [1, 2], "series": {"flow": {"mean": [1, 2], "std": [0.1, 0.2]}},
"rollout_std": [0.1, 0.2], "reference": {"mean": [1.1, 1.9], "std": [0.1, 0.1]},
"reference_mean": [1.1, 1.9],
"reference_std": [0.1, 0.1],
"ylabel": "e", "ylabel": "e",
}, },
), ),
@@ -217,7 +287,7 @@ def test_render_one_of_each_kind(tmp_path: Path):
"species", "species",
{ {
"labels": ["e-", "gamma"], "labels": ["e-", "gamma"],
"rollout": [0.6, 0.4], "series": {"flow": [0.6, 0.4], "wgan": [0.55, 0.45]},
"reference": [0.5, 0.5], "reference": [0.5, 0.5],
"ylabel": "frac", "ylabel": "frac",
}, },
@@ -228,7 +298,22 @@ def test_render_one_of_each_kind(tmp_path: Path):
"single_hist", "single_hist",
"Single", "Single",
"x", "x",
{"edges": [0, 1, 2], "rollout": [5, 1], "log_y": True}, {"edges": [0, 1, 2], "series": {"flow": [5, 1]}, "log_y": True},
),
Reduced(
"hm1",
"secondaries",
"heatmap",
"Heatmap (single rollout)",
"predicted",
{
"series": {"flow": [[1, 0], [0, 1]]},
"reference": [[2, 0], [0, 1]],
"row_labels": ["0", "1+"],
"col_labels": ["0", "1+"],
"cbar_label": "count",
"log_color": True,
},
), ),
] ]
try: try:
@@ -276,8 +361,8 @@ def test_figure_params_v2_basics_and_router_and_epoch():
}, },
"conditioning": {"particle": {"type": "physical"}}, "conditioning": {"particle": {"type": "physical"}},
} }
run_meta = {"training_epoch": 12, "best_val_loss": 0.123456, "steps": 10} meta = {"training_epoch": 12, "best_val_loss": 0.123456, "steps": 10, "model_config": mc}
params = render_mod._figure_params(run_meta | {"model_config": mc}) params = render_mod._figure_params({"rollouts": {"rollout": meta}})
assert params == { assert params == {
"hidden_dim": 256, "hidden_dim": 256,
"n_res_blocks": 4, "n_res_blocks": 4,
@@ -297,8 +382,8 @@ def test_figure_params_v2_wgan_reports_noise_dim_not_steps():
"wgan": {"noise_dim": 32}, "wgan": {"noise_dim": 32},
}, },
} }
run_meta = {"model_config": mc, "steps": 10} meta = {"model_config": mc, "steps": 10}
params = render_mod._figure_params(run_meta) params = render_mod._figure_params({"rollouts": {"rollout": meta}})
assert params["mode"] == "wgan" assert params["mode"] == "wgan"
assert params["noise_dim"] == 32 assert params["noise_dim"] == 32
assert "steps" not in params assert "steps" not in params
@@ -309,18 +394,18 @@ def test_figure_params_v2_reports_mode_s2_only_when_it_differs():
"stage1_model": {"generator": "flow"}, "stage1_model": {"generator": "flow"},
"stage2_model": {"generator": "flow"}, "stage2_model": {"generator": "flow"},
} }
assert "mode_s2" not in render_mod._figure_params({"model_config": same}) assert "mode_s2" not in render_mod._figure_params({"rollouts": {"rollout": {"model_config": same}}})
mixed = { mixed = {
"stage1_model": {"generator": "flow"}, "stage1_model": {"generator": "flow"},
"stage2_model": {"generator": "wgan"}, "stage2_model": {"generator": "wgan"},
} }
params = render_mod._figure_params({"model_config": mixed}) params = render_mod._figure_params({"rollouts": {"rollout": {"model_config": mixed}}})
assert params["mode_s2"] == "wgan" assert params["mode_s2"] == "wgan"
def test_figure_params_old_shape_basics(): def test_figure_params_old_shape_basics():
run_meta = { meta = {
"model_config": { "model_config": {
"hidden_dim": 128, "hidden_dim": 128,
"n_blocks": 3, "n_blocks": 3,
@@ -332,7 +417,7 @@ def test_figure_params_old_shape_basics():
"best_val_loss": 0.5, "best_val_loss": 0.5,
"steps": 20, "steps": 20,
} }
params = render_mod._figure_params(run_meta) params = render_mod._figure_params({"rollouts": {"rollout": meta}})
assert params == { assert params == {
"hidden_dim": 128, "hidden_dim": 128,
"n_blocks": 3, "n_blocks": 3,
@@ -346,20 +431,30 @@ def test_figure_params_old_shape_basics():
def test_figure_params_old_shape_wgan_reports_noise_dim_not_steps(): def test_figure_params_old_shape_wgan_reports_noise_dim_not_steps():
run_meta = { meta = {
"model_config": {"mode": "wgan", "noise_dim": 16}, "model_config": {"mode": "wgan", "noise_dim": 16},
"steps": 20, "steps": 20,
} }
params = render_mod._figure_params(run_meta) params = render_mod._figure_params({"rollouts": {"rollout": meta}})
assert params["noise_dim"] == 16 assert params["noise_dim"] == 16
assert "steps" not in params assert "steps" not in params
def test_figure_params_multi_rollout_names_the_series():
run_meta = {"rollouts": {"flow": {"model_config": {"mode": "flow"}}, "wgan": {"model_config": {"mode": "wgan"}}}}
assert render_mod._figure_params(run_meta) == {"rollouts": "flow, wgan"}
def test_figure_params_empty_rollouts_is_empty():
assert render_mod._figure_params({}) == {}
assert render_mod._figure_params({"rollouts": {}}) == {}
def test_plot_metadata_includes_note_and_run_meta_parameters(): def test_plot_metadata_includes_note_and_run_meta_parameters():
r = Reduced("u", "router", "unavailable", "Unavailable", "x", {"note": "no router data"}) r = Reduced("u", "router", "unavailable", "Unavailable", "x", {"note": "no router data"})
meta = render_mod._plot_metadata(r, {"title": "run-1", "checkpoint": "ckpt.pt"}) meta = render_mod._plot_metadata(r, {"title": "run-1", "reference": "ref.parquet", "rollouts": {"rollout": {}}})
assert meta["note"] == "no router data" assert meta["note"] == "no router data"
assert meta["parameters"] == {"checkpoint": "ckpt.pt"} assert meta["parameters"] == {"reference": "ref.parquet", "rollouts": {"rollout": {}}}
assert "title" not in meta["parameters"] assert "title" not in meta["parameters"]
@@ -368,3 +463,30 @@ def test_plot_metadata_omits_parameters_when_run_meta_empty():
meta = render_mod._plot_metadata(r, {}) meta = render_mod._plot_metadata(r, {})
assert "parameters" not in meta assert "parameters" not in meta
assert "note" not in meta assert "note" not in meta
def test_tex_escape_handles_percent_and_other_special_chars():
assert render_mod._tex_escape("90% of deposited energy") == r"90\% of deposited energy"
assert render_mod._tex_escape(r"a_b & c#d $e {f} \bar") == r"a\_b \& c\#d \$e \{f\} \textbackslash{}bar"
def test_render_survives_title_and_xlabel_with_literal_percent(tmp_path: Path):
# Regression test for gitea #81: a literal "%" in a catalog title (e.g.
# "Shower containment depth (90% of deposited energy)") crashed the whole
# LaTeX render, since usetex treats an unescaped "%" as a comment marker.
reduced = [
Reduced(
"shower_containment_depth_90",
"shower",
"single_hist",
"Shower containment depth (90% of deposited energy)",
"depth containing 90% of deposited energy [mm]",
{"edges": [0, 1, 2], "series": {"flow": [5, 1]}},
),
]
try:
pdfs = _try_render(reduced, tmp_path)
except RuntimeError as e: # LaTeX missing at render time
pytest.skip(f"LaTeX rendering unavailable: {e}")
assert len(pdfs) == 1
assert pdfs[0].exists()
+52 -12
View File
@@ -10,7 +10,9 @@ from giant.analysis.router_gating import (
compute_router_gating, compute_router_gating,
compute_router_share_by_pdg, compute_router_share_by_pdg,
compute_router_share_by_process, compute_router_share_by_process,
compute_router_specialization,
) )
from giant.analysis.sources import RolloutSide
from giant.data.transforms import Normalizer from giant.data.transforms import Normalizer
from giant.model.network import build_models from giant.model.network import build_models
@@ -34,7 +36,7 @@ def _model_cfg() -> dict:
} }
def _write_checkpoint(tmp_path) -> str: def _write_checkpoint(tmp_path, name: str = "ckpt.pt") -> str:
cfg = _model_cfg() cfg = _model_cfg()
stage1 = build_models(cfg)["stage1"] stage1 = build_models(cfg)["stage1"]
assert stage1 is not None assert stage1 is not None
@@ -48,7 +50,7 @@ def _write_checkpoint(tmp_path) -> str:
"mat_map": _MAT_MAP, "mat_map": _MAT_MAP,
"normalizer": {"cond": norm.to_dict()}, "normalizer": {"cond": norm.to_dict()},
} }
path = tmp_path / "ckpt.pt" path = tmp_path / name
torch.save(ckpt, path) torch.save(ckpt, path)
return str(path) return str(path)
@@ -86,42 +88,80 @@ def _steps_frame(process: bool = False) -> pl.LazyFrame:
return pl.DataFrame(data).lazy() return pl.DataFrame(data).lazy()
def _side(checkpoint: str | None, lf: pl.LazyFrame) -> RolloutSide:
return RolloutSide(all=lf, phys=lf, checkpoint=checkpoint)
def test_compute_router_gating_shapes(tmp_path): def test_compute_router_gating_shapes(tmp_path):
checkpoint = _write_checkpoint(tmp_path) checkpoint = _write_checkpoint(tmp_path)
lf = _steps_frame() lf = _steps_frame()
r = compute_router_gating(checkpoint, lf, lf) r = compute_router_gating({"rollout": _side(checkpoint, lf)}, lf)
assert r.kind == "router_gating" assert r.kind == "router_gating"
assert r.payload["n_experts"] == 2 assert list(r.payload["series"]) == ["rollout"]
entry = r.payload["series"]["rollout"]
assert entry["n_experts"] == 2
for side in ("rollout", "reference"): for side in ("rollout", "reference"):
means = r.payload[side]["means"] means = entry[side]["means"]
assert means, f"{side} produced no bins" assert means, f"{side} produced no bins"
assert all(abs(sum(row) - 1.0) < 1e-5 for row in means) assert all(abs(sum(row) - 1.0) < 1e-5 for row in means)
def test_compute_router_gating_missing_checkpoint_is_unavailable(): def test_compute_router_gating_missing_checkpoint_is_unavailable():
lf = _steps_frame() lf = _steps_frame()
r = compute_router_gating(None, lf, lf) r = compute_router_gating({"rollout": _side(None, lf)}, lf)
assert r.kind == "unavailable" assert r.kind == "unavailable"
assert "note" in r.payload assert "note" in r.payload
assert r.title assert r.title
def test_compute_router_gating_two_rollouts_only_moe_ones_included(tmp_path):
lf = _steps_frame()
ckpt = _write_checkpoint(tmp_path)
rollouts = {"flow": _side(None, lf), "moe": _side(ckpt, lf)}
r = compute_router_gating(rollouts, lf)
assert list(r.payload["series"]) == ["moe"]
def test_compute_router_specialization_two_rollouts(tmp_path):
lf = _steps_frame()
ckpt_a = _write_checkpoint(tmp_path, "a.pt")
ckpt_b = _write_checkpoint(tmp_path, "b.pt")
rollouts = {"a": _side(ckpt_a, lf), "b": _side(ckpt_b, lf)}
r = compute_router_specialization(rollouts, lf)
assert r.kind == "router_specialization"
assert list(r.payload["series"]) == ["a", "b"]
for entry in r.payload["series"].values():
assert entry["chance_level"] == 0.5
assert len(entry["rollout"]["centers"]) == len(entry["rollout"]["score"])
def test_compute_router_share_by_pdg(tmp_path): def test_compute_router_share_by_pdg(tmp_path):
checkpoint = _write_checkpoint(tmp_path) checkpoint = _write_checkpoint(tmp_path)
lf = _steps_frame() lf = _steps_frame()
r = compute_router_share_by_pdg(checkpoint, lf, lf, top_pdgs=[11, 22]) r = compute_router_share_by_pdg({"rollout": _side(checkpoint, lf)}, lf, top_pdgs=[11, 22])
assert r.kind == "router_share" assert r.kind == "router_share"
entry = r.payload["series"]["rollout"]
for side in ("rollout", "reference"): for side in ("rollout", "reference"):
assert set(r.payload[side]) == {"e-", "gamma"} assert set(entry[side]) == {"e-", "gamma"}
for shares in r.payload[side].values(): for shares in entry[side].values():
assert abs(sum(shares) - 1.0) < 1e-5 assert abs(sum(shares) - 1.0) < 1e-5
def test_compute_router_share_by_process(tmp_path): def test_compute_router_share_by_process(tmp_path):
checkpoint = _write_checkpoint(tmp_path) checkpoint = _write_checkpoint(tmp_path)
lf = _steps_frame(process=True) lf = _steps_frame(process=True)
r = compute_router_share_by_process(checkpoint, lf) r = compute_router_share_by_process({"rollout": _side(checkpoint, lf)}, lf)
assert r.kind == "router_share" assert r.kind == "router_share"
assert set(r.payload["categories"]) <= {"eIoni", "compt"} entry = r.payload["series"]["rollout"]
for shares in r.payload["reference"].values(): assert set(entry["categories"]) <= {"eIoni", "compt"}
for shares in entry["reference"].values():
assert abs(sum(shares) - 1.0) < 1e-5 assert abs(sum(shares) - 1.0) < 1e-5
def test_no_moe_rollouts_are_unavailable(tmp_path):
lf = _steps_frame()
rollouts = {"flow": _side(None, lf), "wgan": _side(None, lf)}
assert compute_router_gating(rollouts, lf).kind == "unavailable"
assert compute_router_share_by_pdg(rollouts, lf, top_pdgs=[11, 22]).kind == "unavailable"
assert compute_router_share_by_process(rollouts, lf).kind == "unavailable"
assert compute_router_specialization(rollouts, lf).kind == "unavailable"
+333
View File
@@ -0,0 +1,333 @@
"""Tests for giant.training.plots (gitea #75) — render smoke tests skipped
where plotstyle/LaTeX is unavailable, plus pure-function column-classification
coverage that needs neither."""
from __future__ import annotations
import csv
from pathlib import Path
import pytest
pytest.importorskip("plotstyle")
from giant.training import plots as plots_mod # noqa: E402
from giant.training.plots import MetricsTable, derive_metrics_dir, render_metrics # noqa: E402
# --- fixtures ----------------------------------------------------------
_RICH_HEADER = [
"epoch",
"stage1/train/loss",
"stage1/train/loss_gen",
"stage1/train/nsec_acc",
"stage1/train/grad_norm",
"stage1/val/loss",
"stage1/val/loss_gen",
"stage1/val/nsec_acc",
"stage1/lr",
"stage1/router/entropy",
"stage1/router/util_min",
"stage1/router/util_max",
"stage1/router/util_std",
"stage2/train/d_loss",
"stage2/train/g_loss",
"stage2/train/wasserstein",
"stage2/train/gp_loss",
"stage2/train/loss_nsec",
"stage2/train/nsec_acc",
"stage2/train/grad_norm_d",
"stage2/train/grad_norm_g",
"stage2/lr",
"stage2/critic_lr",
"val/loss",
"val/marginal_kl",
"grad_norm",
"gpu_mem_mb",
"samples_per_sec",
"is_best",
"epoch_time_s",
]
_RICH_ROWS = [
[
1,
1.0,
0.8,
0.5,
1.2,
0.9,
0.7,
0.6,
3e-4,
1.5,
0.05,
0.3,
0.1,
-0.2,
0.3,
0.5,
0.1,
0.4,
0.4,
0.9,
1.1,
3e-4,
1e-4,
0.85,
0.4,
2.1,
512.0,
100.0,
1,
5.0,
],
[
2,
0.8,
0.6,
0.6,
1.0,
0.7,
0.5,
0.7,
2e-4,
1.6,
0.06,
0.28,
0.09,
-0.1,
0.25,
0.4,
0.09,
0.3,
0.5,
0.8,
1.0,
2e-4,
8e-5,
0.7,
0.35,
1.9,
520.0,
105.0,
0,
5.1,
],
]
_MINIMAL_HEADER = [
"epoch",
"stage1/train/loss",
"stage1/train/loss_gen",
"stage1/val/loss",
"stage1/val/loss_gen",
"stage1/lr",
"val/loss",
"grad_norm",
"gpu_mem_mb",
"samples_per_sec",
"is_best",
"epoch_time_s",
]
_MINIMAL_ROWS = [
[1, 1.0, 0.8, 0.9, 0.7, 3e-4, 0.85, 0.4, 0.0, 100.0, 0, 5.0],
[2, 0.8, 0.6, 0.7, 0.5, 2e-4, 0.7, 0.35, 0.0, 105.0, 1, 5.1],
]
def _write_csv(path: Path, header: list[str], rows: list[list]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", newline="") as f:
writer = csv.writer(f)
writer.writerow(header)
writer.writerows(rows)
# --- MetricsTable --------------------------------------------------------
def test_metrics_table_load_round_trips(tmp_path: Path):
csv_path = tmp_path / "metrics.csv"
_write_csv(csv_path, _MINIMAL_HEADER, _MINIMAL_ROWS)
table = MetricsTable.load(csv_path)
assert table.epochs == [1, 2]
assert table.columns["stage1/train/loss"] == [1.0, 0.8]
assert "epoch" not in table.columns
assert table.best_epochs() == [2]
def test_metrics_table_best_epochs_empty_without_is_best_column():
table = MetricsTable(epochs=[1, 2], columns={"stage1/train/loss": [1.0, 0.5]})
assert table.best_epochs() == []
# --- column classification (pure functions, no matplotlib) --------------
def _rich_columns() -> dict[str, list]:
return {name: [0.0] for name in _RICH_HEADER if name != "epoch"}
def test_stages_detects_only_stages_present():
assert plots_mod._stages(_rich_columns()) == ["stage1", "stage2"]
assert plots_mod._stages({"stage2/train/loss": [0.0]}) == ["stage2"]
assert plots_mod._stages({"val/loss": [0.0]}) == []
def test_split_matches_stage_and_split_prefix_only():
cols = _rich_columns()
train = plots_mod._split(cols, "stage1", "train")
assert train == {
"loss": "stage1/train/loss",
"loss_gen": "stage1/train/loss_gen",
"nsec_acc": "stage1/train/nsec_acc",
"grad_norm": "stage1/train/grad_norm",
}
assert plots_mod._split(cols, "stage2", "val") == {}
def test_point_in_time_excludes_train_val_router():
cols = _rich_columns()
pit = plots_mod._point_in_time(cols, "stage1")
assert pit == {"lr": "stage1/lr"}
pit2 = plots_mod._point_in_time(cols, "stage2")
assert pit2 == {"lr": "stage2/lr", "critic_lr": "stage2/critic_lr"}
def test_router_columns():
cols = _rich_columns()
assert plots_mod._router(cols, "stage1") == {
"entropy": "stage1/router/entropy",
"util_min": "stage1/router/util_min",
"util_max": "stage1/router/util_max",
"util_std": "stage1/router/util_std",
}
assert plots_mod._router(cols, "stage2") == {}
def test_run_level_excludes_stage_prefixed_columns_including_val_loss_lookalike():
cols = _rich_columns()
run_level = plots_mod._run_level(cols)
assert set(run_level) == {
"val/loss",
"val/marginal_kl",
"grad_norm",
"gpu_mem_mb",
"samples_per_sec",
"is_best",
"epoch_time_s",
}
# stage-prefixed "val/loss" lookalike (stage1/val/loss) must not leak in
assert "stage1/val/loss" not in run_level
def test_loss_keys_excludes_acc_and_wgan_and_grad_norm():
train = {"loss": "x", "loss_gen": "x", "nsec_acc": "x", "grad_norm": "x", "d_loss": "x"}
val = {"loss": "x", "loss_gen": "x"}
assert plots_mod._loss_keys(train, val) == ["loss", "loss_gen"]
# --- derive_metrics_dir ---------------------------------------------------
def test_derive_metrics_dir_explicit_out_dir_wins():
assert derive_metrics_dir("runs/my-run", out_dir="/somewhere") == Path("/somewhere")
def test_derive_metrics_dir_default_base():
assert derive_metrics_dir("runs/my-run", default_base="/data/analysis_runs") == Path(
"/data/analysis_runs/metrics_my-run"
)
def test_derive_metrics_dir_falls_back_to_cwd_analysis_runs(monkeypatch, tmp_path):
monkeypatch.chdir(tmp_path)
assert derive_metrics_dir("runs/my-run") == tmp_path / "analysis_runs" / "metrics_my-run"
# --- render_metrics end to end -------------------------------------------
def _try_render(run_dir: Path, out_dir: Path) -> list[Path]:
try:
return render_metrics(run_dir, out_dir)
except RuntimeError as e: # LaTeX missing at render time
pytest.skip(f"LaTeX rendering unavailable: {e}")
def test_render_metrics_rich_run_produces_expected_plots_outside_run_dir(tmp_path: Path):
run_dir = tmp_path / "run"
out_dir = tmp_path / "out"
_write_csv(run_dir / "metrics.csv", _RICH_HEADER, _RICH_ROWS)
paths = _try_render(run_dir, out_dir)
names = {p.stem for p in paths}
assert names == {
"overview",
"stage1_loss",
"stage2_loss",
"lr",
"stage1_accuracy",
"stage2_accuracy",
"grad_norm",
"stage1_router",
"stage2_wgan_balance",
"throughput",
}
assert all(p.exists() for p in paths)
assert all(p.is_relative_to(out_dir) for p in paths)
# nothing written into the training run directory itself
assert not any(run_dir.rglob("*.pdf"))
def test_render_metrics_minimal_run_omits_router_wgan_accuracy(tmp_path: Path):
run_dir = tmp_path / "run"
out_dir = tmp_path / "out"
_write_csv(run_dir / "metrics.csv", _MINIMAL_HEADER, _MINIMAL_ROWS)
paths = _try_render(run_dir, out_dir)
names = {p.stem for p in paths}
assert names == {"overview", "stage1_loss", "lr", "grad_norm", "throughput"}
assert "stage1_accuracy" not in names
assert "stage1_router" not in names
assert "stage1_wgan_balance" not in names
def test_render_metrics_default_out_dir_uses_analysis_runs_convention(tmp_path: Path):
run_dir = tmp_path / "runs" / "my-run"
_write_csv(run_dir / "metrics.csv", _MINIMAL_HEADER, _MINIMAL_ROWS)
default_base = tmp_path / "analysis_runs"
try:
paths = render_metrics(run_dir, default_base=default_base)
except RuntimeError as e:
pytest.skip(f"LaTeX rendering unavailable: {e}")
assert paths
assert all(p.is_relative_to(default_base / "metrics_my-run") for p in paths)
# --- CLI -------------------------------------------------------------------
def test_cli_analyze_metrics_smoke(tmp_path: Path):
from typer.testing import CliRunner
from giant.cli import app
run_dir = tmp_path / "run"
out_dir = tmp_path / "out"
_write_csv(run_dir / "metrics.csv", _MINIMAL_HEADER, _MINIMAL_ROWS)
runner = CliRunner()
result = runner.invoke(app, ["analyze", "metrics", str(run_dir), "--out", str(out_dir)])
if result.exit_code != 0 and "latex" in (str(result.output) + str(result.exception)).lower():
pytest.skip("LaTeX rendering unavailable")
assert result.exit_code == 0, result.output or result.exception
assert any(out_dir.glob("*.pdf"))
+30 -6
View File
@@ -3,6 +3,9 @@
from __future__ import annotations from __future__ import annotations
import polars as pl
from giant.analysis.sources import RolloutSide
from giant.analysis.type_embedding_distance import compute_type_embedding_l1_distance from giant.analysis.type_embedding_distance import compute_type_embedding_l1_distance
@@ -18,26 +21,47 @@ def _summary(n=100):
} }
def _side(l1_dist: dict | None) -> RolloutSide:
empty = pl.LazyFrame()
return RolloutSide(all=empty, phys=empty, type_embedding_l1_dist=l1_dist)
def test_none_is_unavailable(): def test_none_is_unavailable():
r = compute_type_embedding_l1_distance(None) r = compute_type_embedding_l1_distance({"rollout": _side(None)})
assert r.kind == "unavailable" assert r.kind == "unavailable"
assert r.id == "type_embedding_l1_distance" assert r.id == "type_embedding_l1_distance"
assert r.payload["note"] assert r.payload["note"]
def test_summary_produces_single_hist(): def test_summary_produces_single_hist():
r = compute_type_embedding_l1_distance(_summary()) r = compute_type_embedding_l1_distance({"rollout": _side(_summary())})
assert r.kind == "single_hist" assert r.kind == "single_hist"
assert r.id == "type_embedding_l1_distance" assert r.id == "type_embedding_l1_distance"
assert r.payload["edges"] == [0.0, 1.0, 2.0, 3.0] assert r.payload["edges"] == [0.0, 1.0, 2.0, 3.0]
assert r.payload["rollout"] == [30, 40, 30] assert r.payload["series"]["rollout"] == [30, 40, 30]
assert r.payload["log_x"] is True assert r.payload["log_x"] is True
assert r.payload["log_y"] is True assert r.payload["log_y"] is True
assert "n=100" in r.payload["note"] assert "n=100" in r.payload["note"]
def test_single_hist_payload_shape_matches_render_contract(): def test_single_hist_payload_shape_matches_render_contract():
"""_render_single (giant.analysis.render) requires len(rollout) == """_render_single (giant.analysis.render) requires each series' length ==
len(edges) - 1.""" len(edges) - 1."""
r = compute_type_embedding_l1_distance(_summary()) r = compute_type_embedding_l1_distance({"rollout": _side(_summary())})
assert len(r.payload["rollout"]) == len(r.payload["edges"]) - 1 assert len(r.payload["series"]["rollout"]) == len(r.payload["edges"]) - 1
def test_two_rollouts_both_populated():
r = compute_type_embedding_l1_distance({"flow": _side(_summary(50)), "wgan": _side(_summary(80))})
assert list(r.payload["series"]) == ["flow", "wgan"]
assert "n=50" in r.payload["note"] and "n=80" in r.payload["note"]
def test_one_of_two_rollouts_populated_only_that_one_appears():
r = compute_type_embedding_l1_distance({"flow": _side(None), "wgan": _side(_summary())})
assert list(r.payload["series"]) == ["wgan"]
def test_none_populated_across_rollouts_is_unavailable():
r = compute_type_embedding_l1_distance({"flow": _side(None), "wgan": _side(None)})
assert r.kind == "unavailable"
+28 -1
View File
@@ -2,7 +2,7 @@ import torch
from giant.config import ConditioningAxisConfig from giant.config import ConditioningAxisConfig
from giant.constants import COND_DIM, K_MAX, SEC_DIM, SEC_SLOT_DIM, X_DIM from giant.constants import COND_DIM, K_MAX, SEC_DIM, SEC_SLOT_DIM, X_DIM
from giant.model.network import CriticModel, Stage1Model, Stage2OneShot from giant.model.network import CriticModel, LinearTrunk, Stage1Model, Stage2OneShot
from giant.model.wgan import critic_loss, generator_loss, gradient_penalty from giant.model.wgan import critic_loss, generator_loss, gradient_penalty
from giant.sample import sample_secondaries_wgan, sample_wgan from giant.sample import sample_secondaries_wgan, sample_wgan
@@ -115,6 +115,33 @@ def test_critic_output_shape():
assert out.shape == (B,) assert out.shape == (B,)
def test_critic_model_honours_trunk_type_and_block_conditioning():
"""gitea #57: CriticModel routes its body through build_trunk/build_block
like every generator stage model, instead of hand-rolling a plain
ResBlock stack."""
B = 8
critic = CriticModel(
pdg_vocab=3,
mat_vocab=2,
particle_cfg=PARTICLE_CFG,
material_cfg=MATERIAL_CFG,
in_dim=X_DIM,
hidden_dim=32,
n_res_blocks=2,
stage="stage1",
trunk_type="linear",
block_conditioning="adaln",
)
assert isinstance(critic.trunk, LinearTrunk)
cond_cont, cond_cat = _cond(B)
real = torch.randn(B, X_DIM)
fake = torch.randn(B, X_DIM)
loss = critic_loss(lambda x: critic(x, cond_cont, cond_cat), real, fake.detach(), gp_weight=10.0)
loss.backward()
for name, p in critic.named_parameters():
assert p.grad is not None, f"no grad for {name}"
def test_sample_wgan_shape(): def test_sample_wgan_shape():
B = 6 B = 6
model = _small_generator() model = _small_generator()
Generated
+1 -1
View File
@@ -675,7 +675,7 @@ wheels = [
[[package]] [[package]]
name = "giant" name = "giant"
version = "0.3.5" version = "0.3.11"
source = { editable = "." } source = { editable = "." }
dependencies = [ dependencies = [
{ name = "numpy" }, { name = "numpy" },