torch/pandas/pyarrow/polars/uproot/awkward/particle were all imported
at module scope in giant/cli.py and giant/tools/dwarf.py, so even
`--help` paid ~1.6-1.9s of import cost. Move those imports into the
command bodies that actually need them (following the deferred-import
pattern already used for analysis/render/plots/sklearn/wandb), cutting
`giant --help` to ~0.3s and `dwarf --help` to ~0.2s with no change to
any command's actual behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A bare "pull_request:" key parses to null in YAML, which the runner
apparently doesn't treat as "trigger with defaults" the way an empty
mapping does — the open PR for this branch got no CI run at all.
- Drop [skip ci] from the bump-version/changelog/tag-sync commits so
master's tip always has a check run instead of only the merge commit.
- Filter those chore commits out of the changelog via message pattern
instead of the now-removed [skip ci] tag.
- Only run CI on push to master (plus tags); pull requests to any branch
still run the full suite.
- Add a publish-package job that builds and publishes to the Gitea PyPI
registry on every tag push, after tests and version sync pass.
giant predict/rollout rebuilt models straight from ckpt["model_config"] with
no way to change sampling-only keys (e.g. stage2_model.n_sec.stop_sampling)
without retraining. Adds config_overrides to load_for_inference, validated
against giant.config.INFERENCE_OVERRIDES so a typo or shape-bearing key
raises CheckpointCompatibilityError up front instead of an opaque
load_state_dict mismatch. Wired as a repeatable --set dotted.path=value on
both CLI commands, recorded in the rollout YAML sidecar, and surfaced in
`giant model summary`'s output.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Taking argmax over the n_sec classifier logits collapses secondary
multiplicity onto its conditional mode at fixed pre-step conditioning,
under-dispersing n_sec in rollouts and biasing low wherever the true
conditional count distribution is right-skewed (typical for
multiplicity).
Generalizes stage2_model.n_sec.stop_sampling (previously stop_token-only)
into stage2_model.n_sec.sampling, covering both "head" (greedy: argmax;
sample: categorical draw via torch.multinomial) and "stop_token" (unchanged:
greedy threshold / Bernoulli draw) modes. stop_sampling is kept as a
deprecated alias in NSecConfig.from_dict and migrate_config, since it
appears in existing checkpoints' model_config. Default stays "greedy" so
existing runs/checkpoints are unaffected.
Replace the event-level n_sec confusion matrix with two step-resolved
secondary-multiplicity comparisons:
- sec_count_per_step: overlay histogram of how many secondaries a single
step emits, rollout series vs reference.
- sec_count_per_step_by_species: heatmap of per-step multiplicity of one
species (zero row included) against species, drawn as one panel per
rollout plus a reference panel, raw counts on a log color scale.
Both are backed by a new sources.secondaries_by_step view, which tags each
secondary with its emitting step — (event_id, parent_id, birth position)
on the rollout side, the row index on the reference side — so neither plot
needs a join against the step frame. Steps that emitted nothing are
recovered by subtraction from the chunk's step count, keeping both specs
sum-mergeable across condor chunks.
The rollout multiplicity is derived from the actual secondary birth rows
rather than the n_sec_pred column, which records the predicted count
before the per-event max-tracks cap.
_render_heatmap gained reference-panel and log-color support;
marginal_distance_summary sets neither key and is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLAUDE.md still described the pre-v0.3.0 codebase: the Stage-2
autoregressive redesign as "designed, not implemented", a monolithic
network.py, a single global model.conditioning switch, and WGAN as
"implemented, not yet tested".
- Architecture rewritten around the actual giant/model split
(layers/encoders/trunks/routers/history/objectives/models/builders/
_legacy/summary; network.py is now a re-export shim), plus
cond_layout.py, checkpoint_io.py, _migration.py, data/setup_cache.py
and giant/training/.
- Conditioning documented per axis (conditioning.particle /
conditioning.material, each physical|embedding|onehot, freely mixed).
- Stage 2 documented with both decoders, n_sec.mode, teacher forcing,
stage1_context and the three particle_type.target options.
- Roadmap: v0.3.0 recorded as implemented/released; WGAN and MoE routing
as implemented but unvalidated, with the router retrain as next step.
- Analysis: run dir is <cwd>/analysis_runs/analysis_<id>, plus
variables/reduced/runtime_estimate and analyze list/merge-one/metrics.
- Added giant model summary, configs/, and the CI-automated version and
changelog bump.
README drift fixes only: project tree for the model/analysis/training
splits, analyze run-dir default, missing subcommands, --precision and
--stage2-stage1-context, the extras list, and two accuracy fixes
(--router configures stage 1 only; --conditioning sets two independent
axes at once).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The automated changelog (gitea #50) deliberately started fresh with no
backfill; this reverses that call now that it's wanted. v0.2.0-v0.3.2 are
generated from tag history via git-cliff/cliff.toml, matching the format of
existing entries. v0.3.3 was bumped but never tagged, so its commits stay
folded into the existing v0.3.4 entry. The v0.2.0 range (198 uncurated
pre-automation commits) is hand-curated to drop duplicate commits and
dev-log noise (WIP markers, incomplete-validation runs, repeated
"Apply ruff format").
shower_containment_depth_90/95's title contains a literal "%" (e.g.
"...(90% of deposited energy)"), which usetex reads as a comment marker
and aborts LaTeX compilation. Since render_all processes reduced JSON
files in sorted filename order, this killed every plot id sorting after
these two in the same run.
Escape title/xlabel once, centrally, in render()'s dispatch (the one
place every renderer kind draws them from before handing off to
plotstyle/matplotlib) rather than at each catalog.py call site, so any
future catalog title with a %, &, #, etc. is covered automatically.
_plot_metadata keeps using the unescaped Reduced for the gallery YAML,
since that's not LaTeX.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
giant analyze compares N rollout YAMLs against one shared reference file
(all must name the same dataset, checked up front) instead of exactly one
rollout vs one reference, rendering each rollout as its own colored series
against a single reference line/panel. Series names come from a repeated
--label flag, else the YAML stem, else "rollout" for a single YAML — a
single-rollout run keeps rendering identically to before this change.
Bundle now holds a name-keyed dict of rollout sides instead of one fixed
pair, every catalog compute_partial/finalize builds a Reduced.payload
keyed the same way ("series": {name: ...}, "reference": ... as the one
distinguished non-rollout entry), and every renderer draws N series (or
N panels, for the two heatmap-shaped specs and the router/type-embedding
diagnostics, which are inherently one-matrix/one-checkpoint per rollout)
against the reference's fixed dashed-ink style.
CliRunner stores an uncaught exception in result.exception, not
result.output, so the skip condition never matched and the test
failed outright on CI machines without LaTeX installed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Picks 4 of the 7 catalog additions the issue proposed (the smaller-lift
ones; 2D joint plots, PIT calibration, and the throughput/accuracy scatter
are left for follow-up issues):
- marginal_distance_summary: a var x grouping-axis KS-statistic heatmap,
reusing the existing marginal hist1d compute and just adding a finalize —
a single at-a-glance regression scorecard instead of N overlay plots.
- n_sec_confusion: predicted (rollout) vs true (reference) secondary count
per event, paired by event_id since a rollout is seeded from the same
events as its reference file. Needed a new zero-filling primitive
(reduce.sec_count_by_event) since a plain group_by over secondary rows
silently drops zero-secondary events.
- shower_containment_depth_{90,95}: per-event depth containing 90%/95% of
deposited energy, derived from the same per-event depth-bin matrix the
longitudinal profile already computes.
- router_specialization: max gate weight vs energy per side, summarizing
router_gating's full stacked area into the one trend line the roadmap's
MoE writeup describes (the ~60-65% ceiling), to make a future
lambda_balance>0 retrain's effect on specialization checkable at a glance.
Both new heatmap-shaped plots (distance summary, confusion matrix) share one
new "heatmap" Reduced kind/renderer rather than two near-identical ones.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MetricsCollector writes one row per epoch to <run_dir>/metrics.csv, but
nothing read or plotted it. giant/training/plots.py reads the CSV header
dynamically (the column set varies by run: flow/ddpm vs wgan, routed vs
not) and renders loss/lr/accuracy/grad-norm/router/wgan-balance/throughput
plots with the same plotstyle conventions giant/analysis/render.py uses,
skipping any figure whose columns aren't present for a given run.
Wired up as `giant analyze metrics <run_dir>`, writing PDFs into the same
gitignored analysis_runs/ directory `analyze prep`/`submit` already use
(derive_metrics_dir mirrors derive_run_dir) rather than into the training
run directory itself.
CriticModel was the one stage-shaped class left out of the trunk-registry
(gitea #33), block-conditioning-registry (gitea #34), and StageModel-base
(gitea #39) refactors: it hand-rolled a plain ResBlock stack, so a
routed/FiLM/AdaLN trunk was available to every generative stage model except
the critic competing against them under WGAN-GP.
CriticModel now subclasses StageModel (reusing its cond_enc construction, and
a stage-2 context-fusion helper factored out of Stage2OneShot onto the base)
and builds its body via build_trunk (output width 1) instead of a bespoke
ResBlock loop, so trunk.type/trunk.block_conditioning now affect the critic
too. Each stage's critic inherits its own generator's trunk config rather
than a new critic_trunk config key, mirroring the existing
critic_hidden_dim/critic_n_res_blocks "0 = inherit from generator" pattern.
Router mixing (MoE) for the critic stays out of scope. Since CriticModel is
training-only and never persisted for inference, and WGAN-GP is still
unbenchmarked, its state_dict shape has no back-compat burden.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
gitea-actions
lars