giant/CHANGELOG.md

Changelog

[0.3.10] - 2026-08-26

Changed

• Backfill CHANGELOG.md for v0.2.0-v0.3.2

• Docs: bring README and CLAUDE.md in line with v0.3.9

[0.3.9] - 2026-08-24

Added

• Add multi-rollout support to giant analyze gitea #77

Changed

• Escape LaTeX-special characters in plot titles/xlabels gitea #81

[0.3.8] - 2026-08-24

Added

• Add giant analyze metrics plots for training progress gitea #75

Fixed

• Fix LaTeX-unavailable skip check in analyze metrics smoke test

[0.3.7] - 2026-08-24

Added

• Add rollout-quality distance, confusion, containment and router plots gitea #76

[0.3.6] - 2026-08-24

Changed

• Give CriticModel a registry-built trunk and StageModel base gitea #57

[0.3.5] - 2026-08-24

Added

• Add "none" variants for router, history, and trunk gitea #45

[0.3.4] - 2026-08-23

Added

• Add giant model summary command gitea #46

• Add per-stage init_from/freeze gitea #42

• Add bf16 autocast to the training loop gitea #47

• Add class-balanced secondary particle-type loss gitea #44

Changed

• Implement stage2_model.stage1_context = "sampled" gitea #41

• Bump patch version to 0.3.3

• Offset event_id across multi-shard reference reads in giant analyze gitea #22

• Auto-bump patch version, tag, and update changelog on merge to master gitea #50

• Document CI_TOKEN's write:repository scope requirement gitea #50

[0.3.2] - 2026-08-17

Added

• Add configs/baseline.toml as the kept reference model

Fixed

• Clamp analysis histogram bins before the i32 cast, not after gitea #61

• Clip raw predicted log_mass in decode_secondaries gitea #54

Changed

• Let dwarf warm-cache take --config so it can't under-warm a config's cache keys gitea #59

• Implement n_sec.mode = "stop_token" for the AR secondary decoder gitea #40

• Bump patch version to 0.3.2

[0.3.1] - 2026-08-14

Added

• Add an Objective registry for the flow/ddpm/wgan generator choice gitea #32

Changed

• Make trunk architecture selectable via a registry gitea #33

• Make ResBlock's conditioning-injection mechanism selectable gitea #34

• Make HistoryEncoder a pluggable registry, like Router/Objective gitea #35

• Deduplicate n_sec_head/type_head MLPs into build_mlp_head gitea #36

• Give the cond_cat/cond_cont column layout one owner gitea #37

• Give Stage1Model/Stage2OneShot/Stage2Autoregressive a shared StageModel base gitea #39

• Pass ConditioningAxisConfig/ParticleTypeConfig themselves instead of raw dicts gitea #38

• Bump patch version to 0.3.1

[0.3.0] - 2026-08-13

Added

• Add v0.3.0 design doc: Stage-2 autoregressive redesign

• Add pytest-cov to dev deps and run coverage in CI

• Add coverage for router-center seeding, geometry batch reader, material topN cache, and setup-cache corruption paths

• Add render.py coverage: figure params, router diagnostics plots, gallery/condor glue

• Add unknown-key validation to config.toml merge (issues.md Issue 2)

• Add consumed-keys audit test (issues.md Issue 5)

Fixed

• Fix test_render_all_run_gallery_invokes_subprocess clobbering LaTeX's own subprocess.run

Removed

• Remove issues.md

Changed

• Refine v0.3.0 design: defaults, deferred scope, open questions

• Document the differentiability position and its validation obligation

• V0.3.0 step 1: new nested config schema, v0.2 migration shim

• V0.3.0 step 2: network.py refactor to composable stage models

• V0.3.0 step 3: per-stage train.py trainers + pipeline.py/cli.py rewrite

• V0.3.0 step 4: type map + particle_type.target = "onehot"/"embedding"

• V0.3.0 step 5: Stage2Autoregressive (history=markov) + §11.4 grad instrumentation

• V0.3.0 step 6: sample.py/rollout.py AR generation + class->PDG decode

• V0.3.0 step 7: AttentionHistory (KV-cached) + scheduled/never teacher forcing

• V0.3.0 post-implementation audit: resolve all 9 tracked discrepancies

• Refactor train.py into giant/training/ around a metrics collector

• Silence the fork-safety warning from num_workers>0 pipeline tests

• Deduplicate giant/training/trainers.py shared per-stage logic

• Rewrite README for v0.3.0 architecture, quick start, and data columns

• Bump version to 0.3.0

• Delete docs/v0.3.0-design.md and strip all references to it

• Apply ruff format

• Downgrade coverage-report upload to actions/upload-artifact@v3

• Bump ruff line-length to 120 and reformat

• Make config dataclasses the single source of truth for DEFAULT_CONFIG

• Extract giant train/new-run's CLI override mapping into a table-driven function (issues.md Issues 3 & 4)

• Mark issues.md Issues 3 & 4 as fixed

• Extract predict/rollout's duplicated inference bootstrap into giant.checkpoint_io (issues.md Issue 5)

• Mark issues.md Issue 5 as fixed

• Unify the two v0.2->v0.3 migration surfaces (issues.md Issue 6)

• Type the data/model/training batch contracts with NamedTuples (issues.md Issue 7)

• Split giant/model/network.py into giant/model/ (issues.md Issue 8)

• Move scripts/ to giant/tools/ (issues.md Issue 9)

• Reject stage2_model.stage1_context = 'sampled' as unimplemented (issues.md Issue 1)

• Honour wgan.critic_hidden_dim/critic_n_res_blocks in build_critics gitea #28

• Validate stage2_model.autoregressive.order in validate_config gitea #30

• Decouple secondary-species vocabulary from conditioning.particle.emb_dim gitea #29

• Skip router auxiliary loss compute when their lambda is 0 gitea #31

[0.2.0] - 2026-08-04

Added

• Add CLAUDE.md with architecture overview and dev commands

• Add streaming data pipeline and giant CLI entry point

• Add giant predict command

• Add ROOT-to-parquet conversion script with convert dependency group

• Add post_pos as a model target via travel_dir decomposition

• Add --coord local mode to predict for raw-space prediction debugging

• Add KL divergence to marginal validation and hook it into the training loop

• Add graceful shutdown on SIGINT/SIGTERM

• Add configurable dropout to ResBlocks

• Add giant.analysis module for notebook-based model quality diagnostics

• Add lazy polars I/O and duplicate KL/constraint checks for giant.analysis

• Add ruff and ty as dev dependencies, fix lint/type findings

• Add linear warmup before cosine LR decay

• Add --batch-size auto to estimate batch size from free GPU memory

• Add hyperparameter scan

• Add --batch-size auto to predict, matching train

• Add tqdm progress bar to predict

• Add KL bar plots and sample_frac to load_predicted_local; ignore root parquet scratch files

• Add event-level shower observables to giant.analysis

• Add total length traveled per event to event observables

• Add pdg energy/length contribution pie plots

• Add export script for Tier 4 event-level/pdg-share plots

• Add mean/median deposited energy and step length plots per event

• Add export script for ETP group-update presentation plots

• Add photon edep export scripts and per-step presentation plots

• Add tooling for a versioned geant_steps dataset layout

• Add --copy mode to migrate_geant_steps.py

• Add update-manifest and create-manifest subcommands to bump_dataset_version

• Add --to flag for bump-gen/bump-schema and --gen flag for update-manifest

• Add disk usage summary to dwarf status

• Add file counts and reference tracking to dwarf status

• Add --comment option to predict, recorded in YAML sidecar

• Add energy-conservation PoC ODE-step comparison scripts

• Add autoregressive shower rollout driver

• Add fast slab lookup for the GeometryOracle, replacing knn as the default

• Add load_rollout_vs_truth to compare rollouts against held-out truth data

• Add mixture-of-experts routing prototype for Stage 1 and Stage 2

• Add ProcessRouter for physics-process-based expert gating

• Add PdgRouter for particle-type-based expert gating

• Add ComposedRouter for multi-axis MoE gating

• Add EMA weights, weight decay, step-based LR schedule, and grad-norm logging to training

• Add WGAN-GP mode as a throwaway fast-eval experiment

• Add router gating diagnostic for MoE checkpoints

• Add Gitea Actions CI pipeline

• Add configs for router energy (embedding/physical) and WGAN baseline runs

• Add opt-in Weights & Biases logging for the training loop

• Add test coverage for resolve_expert_dims

• Add regression coverage for vocab/process index-map builders

• Add dwarf warm-cache to precompute the setup-stage sidecar

• Add giant new-run to scaffold a config.toml + run dir ahead of training

• Add learnable per-expert width and shared temperature to EnergyRouter

• Add opt-in straight-through Gumbel-softmax combine weights to MoE router

• Add gumbel router configs sweeping learnable-knob combinations

• Add gumbel/learn_centers/learn_width/learn_temperature to out-dir naming

• Add bigger WGAN config (hidden_dim=512, n_blocks=6)

• Add data-integrity guards against silent NaN/Inf propagation and races

Fixed

• Fix column names to match actual parquet schema

• Fix installed torch version to be compatible with cuda drivers

• Fix miniCaloSim link in README

• Fix giant.analysis import after Phase 2 dataset API changes

• Fix silent failure modes surfaced by extensive code review

• Fix ruff, ty, and pytest failures; apply ruff format

• Clamp n_sec classification label to K_MAX

• Fix rollout edep mismatch and add truth overlay to Tier 4 observables

• Fix crashes in physical-property conditioning edge cases

• Fix router experts silently ignoring --hidden-dim/--n-blocks

• Fix conditioning="physical" so it can actually generalize past training vocab

• Fix training-loop checkpoint/resume and WGAN bugs

• Fix stale-partial reuse and n_chunks mismatch in analysis condor pipeline

• Fix CLI/tooling robustness gaps and dedupe the Conditioning enum

• Fix test_write_submit_requires_synced_venv for active-venv resolution

Removed

• Remove scripts/train.py in favor of the giant train CLI

• Drop orphaned child tracks instead of nulling secondary targets

Changed

• Initial commit: giant surrogate model with two-phase roadmap in README

• Implement Phase 1: full data pipeline, model, training, and config support

• Handle material column as string type

• Rename pre_energy/post_energy columns to pre_E/post_E

• Rename direction columns from pre_dir_x/y/z to pre_dx/dy/dz

• Batch StreamingStepsDataset internally instead of per-row collate

• Dedup training pipeline, add seeding/resume and per-epoch metrics logging

• Split torch into cpu/cuda extras and pin dependency version bounds

• Apply ruff format and document lint/type tooling in CLAUDE.md

• Update README to match current architecture and tooling

• Make sampler step count configurable for validation

• Calibrate auto batch size separately for inference vs training

• Skip rows with unknown PDG codes during predict

• Buffer predict rows across row-group boundaries before inference

• Export plots for knowledge base

• Rework validation notebook with markdown sections and Tier 4 plots

• Allow steps_to_parquet.py to accept multiple ROOT input files

• Encode edep/secondary/post energy as a conservation-constrained simplex

• Expose dataset/conversion scripts as uv entry points

• Restrict holdout overlap check to holdout vs dev/full only

• Route predict output to UUID-named parquet with YAML reference sidecar

• Implement Phase 2: secondary particle prediction

• Unify dataset/tooling scripts into a single dwarf Typer CLI

• Fold --to/--gen dataset-versioning flags into the dwarf CLI

• Prefix default train output dir with current date

• Color-code dwarf status output by tree level

• Show VERSIONS.md reason extracts in dwarf status

• Wire up predict CLI to load and run the Stage-2 sec_decoder

• Wire up n_sec/species/energy-fraction validation for Stage 2

• Detach Stage-2 type-embedding target to stop self-referential collapse

• Weight Stage-2 secondary loss equally between direction and type-embedding dims

• Recalibrate batch-size estimate for the post-Phase-2 model size

• Update CLAUDE.md and README for the implemented Phase 2 model

• Error on missing secondary lists instead of silently zeroing Stage-2 targets

• Derive a unique per-job seed for minicalosim shard generation

• Rescale secondary energies to exactly consume the e_sec budget

• Support --energy-gev in dwarf make-root for the new minicalosim energy arg

• Stream giant rollout output instead of buffering the whole run

• Scale auto batch-size estimate by MoE expert count during training

• Rewrite analysis module as a lean, fully-streaming pipeline

• Reimplement rollout-vs-truth comparison on the streaming analysis module

• Condition on material/particle physical properties instead of learned embeddings

• Ignore the scratchpad working directory

• Quote the on: key in the CI workflow

• Split CI lint stage into parallel jobs

• Rewrite analysis as streaming rollout-vs-reference plotting pipeline

• Analyze: drive prep/submit from the rollout YAML sidecar

• Analyze: show model/training params on rendered figures

• Deps: install plotstyle from git.larsbogner.de package index

• Analyze: drop stale ty:ignore on plotstyle import

• Test: replace prep(**_CTX) splat with a typed _prep helper

• Analyze: add MoE router gating/share diagnostic plots

• Chore: remove stray CUDA sanity script and stale Phase 2 planning doc

• Docs: document compute environment, WGAN/MoE status, and condor-gpu-train-rollout

• Analyze: normalize pdg dtype in open_side to fix rollout/reference concat

• Analyze: chunk per-plot aggregation across HTCondor jobs

• Analyze: expose bin/pdg options on analyze submit

• Analyze: estimate per-job HTCondor walltime from chunk row count

• Analyze: run condor compute jobs via .venv/bin/giant, not uv run

• Analyze: default condor docker image to alma9-gridjob

• Analyze: raise default condor job memory request to 8192 MB

• Analyze: recalibrate condor walltime model from real cluster timings

• Transforms: pad legacy cond normalizers for pre-physical-conditioning checkpoints

• Analyze: default run directory to /analysis_runs, gitignored

• Docs: record first MoE router rollout benchmark result in the roadmap

• Router: seed EnergyRouter centers from data quantiles instead of a fixed linspace

• Docs: note the EnergyRouter centers_init fix in the roadmap

• Analyze: thread full model/training/rollout/dataset params to plots

• Ci: share one uv sync across jobs, gate tests on lint+type-check, sync tag/version on release tags

• Ci: replace unsupported artifact sharing with a bind-mounted uv cache

• Ci: stop setup-uv from overriding UV_CACHE_DIR

• Ci: re-pin UV_CACHE_DIR after setup-uv, which exports its own value regardless of enable-cache

• Ci: set UV_LINK_MODE=copy to silence the cross-filesystem hardlink warning

• Log batch-level metrics to W&B, not just per-epoch summaries

• Log router health, WGAN grad-norm split, n_sec accuracy, GPU/throughput to W&B

• Persist global_step across --resume so W&B step stays monotonic

• Timestamp default checkpoint dir to avoid W&B run-id collisions

• Skip empty-slice mean/std in sec phys validation print

• Speed up giant train's setup stage

• Speed up _WelfordAccumulator's per-chunk update

• Make default checkpoint out_dir name reflect only non-default hyperparams

• Cache giant train's setup stage in a sidecar file

• Pass --seed through to the train/val event split

• Offset event_id per file to avoid cross-file collisions

• Store a quantile grid instead of a raw reservoir sample in the setup cache

• Scope wandb run config to only-active hyperparameters

• Resolve giant condor wrapper from the active venv, not a hardcoded path

• Bump version to 0.2.0