- Drop [skip ci] from the bump-version/changelog/tag-sync commits so
master's tip always has a check run instead of only the merge commit.
- Filter those chore commits out of the changelog via message pattern
instead of the now-removed [skip ci] tag.
- Only run CI on push to master (plus tags); pull requests to any branch
still run the full suite.
- Add a publish-package job that builds and publishes to the Gitea PyPI
registry on every tag push, after tests and version sync pass.
giant predict/rollout rebuilt models straight from ckpt["model_config"] with
no way to change sampling-only keys (e.g. stage2_model.n_sec.stop_sampling)
without retraining. Adds config_overrides to load_for_inference, validated
against giant.config.INFERENCE_OVERRIDES so a typo or shape-bearing key
raises CheckpointCompatibilityError up front instead of an opaque
load_state_dict mismatch. Wired as a repeatable --set dotted.path=value on
both CLI commands, recorded in the rollout YAML sidecar, and surfaced in
`giant model summary`'s output.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Taking argmax over the n_sec classifier logits collapses secondary
multiplicity onto its conditional mode at fixed pre-step conditioning,
under-dispersing n_sec in rollouts and biasing low wherever the true
conditional count distribution is right-skewed (typical for
multiplicity).
Generalizes stage2_model.n_sec.stop_sampling (previously stop_token-only)
into stage2_model.n_sec.sampling, covering both "head" (greedy: argmax;
sample: categorical draw via torch.multinomial) and "stop_token" (unchanged:
greedy threshold / Bernoulli draw) modes. stop_sampling is kept as a
deprecated alias in NSecConfig.from_dict and migrate_config, since it
appears in existing checkpoints' model_config. Default stays "greedy" so
existing runs/checkpoints are unaffected.
Replace the event-level n_sec confusion matrix with two step-resolved
secondary-multiplicity comparisons:
- sec_count_per_step: overlay histogram of how many secondaries a single
step emits, rollout series vs reference.
- sec_count_per_step_by_species: heatmap of per-step multiplicity of one
species (zero row included) against species, drawn as one panel per
rollout plus a reference panel, raw counts on a log color scale.
Both are backed by a new sources.secondaries_by_step view, which tags each
secondary with its emitting step — (event_id, parent_id, birth position)
on the rollout side, the row index on the reference side — so neither plot
needs a join against the step frame. Steps that emitted nothing are
recovered by subtraction from the chunk's step count, keeping both specs
sum-mergeable across condor chunks.
The rollout multiplicity is derived from the actual secondary birth rows
rather than the n_sec_pred column, which records the predicted count
before the per-event max-tracks cap.
_render_heatmap gained reference-panel and log-color support;
marginal_distance_summary sets neither key and is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLAUDE.md still described the pre-v0.3.0 codebase: the Stage-2
autoregressive redesign as "designed, not implemented", a monolithic
network.py, a single global model.conditioning switch, and WGAN as
"implemented, not yet tested".
- Architecture rewritten around the actual giant/model split
(layers/encoders/trunks/routers/history/objectives/models/builders/
_legacy/summary; network.py is now a re-export shim), plus
cond_layout.py, checkpoint_io.py, _migration.py, data/setup_cache.py
and giant/training/.
- Conditioning documented per axis (conditioning.particle /
conditioning.material, each physical|embedding|onehot, freely mixed).
- Stage 2 documented with both decoders, n_sec.mode, teacher forcing,
stage1_context and the three particle_type.target options.
- Roadmap: v0.3.0 recorded as implemented/released; WGAN and MoE routing
as implemented but unvalidated, with the router retrain as next step.
- Analysis: run dir is <cwd>/analysis_runs/analysis_<id>, plus
variables/reduced/runtime_estimate and analyze list/merge-one/metrics.
- Added giant model summary, configs/, and the CI-automated version and
changelog bump.
README drift fixes only: project tree for the model/analysis/training
splits, analyze run-dir default, missing subcommands, --precision and
--stage2-stage1-context, the extras list, and two accuracy fixes
(--router configures stage 1 only; --conditioning sets two independent
axes at once).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The automated changelog (gitea #50) deliberately started fresh with no
backfill; this reverses that call now that it's wanted. v0.2.0-v0.3.2 are
generated from tag history via git-cliff/cliff.toml, matching the format of
existing entries. v0.3.3 was bumped but never tagged, so its commits stay
folded into the existing v0.3.4 entry. The v0.2.0 range (198 uncurated
pre-automation commits) is hand-curated to drop duplicate commits and
dev-log noise (WIP markers, incomplete-validation runs, repeated
"Apply ruff format").
shower_containment_depth_90/95's title contains a literal "%" (e.g.
"...(90% of deposited energy)"), which usetex reads as a comment marker
and aborts LaTeX compilation. Since render_all processes reduced JSON
files in sorted filename order, this killed every plot id sorting after
these two in the same run.
Escape title/xlabel once, centrally, in render()'s dispatch (the one
place every renderer kind draws them from before handing off to
plotstyle/matplotlib) rather than at each catalog.py call site, so any
future catalog title with a %, &, #, etc. is covered automatically.
_plot_metadata keeps using the unescaped Reduced for the gallery YAML,
since that's not LaTeX.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
giant analyze compares N rollout YAMLs against one shared reference file
(all must name the same dataset, checked up front) instead of exactly one
rollout vs one reference, rendering each rollout as its own colored series
against a single reference line/panel. Series names come from a repeated
--label flag, else the YAML stem, else "rollout" for a single YAML — a
single-rollout run keeps rendering identically to before this change.
Bundle now holds a name-keyed dict of rollout sides instead of one fixed
pair, every catalog compute_partial/finalize builds a Reduced.payload
keyed the same way ("series": {name: ...}, "reference": ... as the one
distinguished non-rollout entry), and every renderer draws N series (or
N panels, for the two heatmap-shaped specs and the router/type-embedding
diagnostics, which are inherently one-matrix/one-checkpoint per rollout)
against the reference's fixed dashed-ink style.
CliRunner stores an uncaught exception in result.exception, not
result.output, so the skip condition never matched and the test
failed outright on CI machines without LaTeX installed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Picks 4 of the 7 catalog additions the issue proposed (the smaller-lift
ones; 2D joint plots, PIT calibration, and the throughput/accuracy scatter
are left for follow-up issues):
- marginal_distance_summary: a var x grouping-axis KS-statistic heatmap,
reusing the existing marginal hist1d compute and just adding a finalize —
a single at-a-glance regression scorecard instead of N overlay plots.
- n_sec_confusion: predicted (rollout) vs true (reference) secondary count
per event, paired by event_id since a rollout is seeded from the same
events as its reference file. Needed a new zero-filling primitive
(reduce.sec_count_by_event) since a plain group_by over secondary rows
silently drops zero-secondary events.
- shower_containment_depth_{90,95}: per-event depth containing 90%/95% of
deposited energy, derived from the same per-event depth-bin matrix the
longitudinal profile already computes.
- router_specialization: max gate weight vs energy per side, summarizing
router_gating's full stacked area into the one trend line the roadmap's
MoE writeup describes (the ~60-65% ceiling), to make a future
lambda_balance>0 retrain's effect on specialization checkable at a glance.
Both new heatmap-shaped plots (distance summary, confusion matrix) share one
new "heatmap" Reduced kind/renderer rather than two near-identical ones.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MetricsCollector writes one row per epoch to <run_dir>/metrics.csv, but
nothing read or plotted it. giant/training/plots.py reads the CSV header
dynamically (the column set varies by run: flow/ddpm vs wgan, routed vs
not) and renders loss/lr/accuracy/grad-norm/router/wgan-balance/throughput
plots with the same plotstyle conventions giant/analysis/render.py uses,
skipping any figure whose columns aren't present for a given run.
Wired up as `giant analyze metrics <run_dir>`, writing PDFs into the same
gitignored analysis_runs/ directory `analyze prep`/`submit` already use
(derive_metrics_dir mirrors derive_run_dir) rather than into the training
run directory itself.
CriticModel was the one stage-shaped class left out of the trunk-registry
(gitea #33), block-conditioning-registry (gitea #34), and StageModel-base
(gitea #39) refactors: it hand-rolled a plain ResBlock stack, so a
routed/FiLM/AdaLN trunk was available to every generative stage model except
the critic competing against them under WGAN-GP.
CriticModel now subclasses StageModel (reusing its cond_enc construction, and
a stage-2 context-fusion helper factored out of Stage2OneShot onto the base)
and builds its body via build_trunk (output width 1) instead of a bespoke
ResBlock loop, so trunk.type/trunk.block_conditioning now affect the critic
too. Each stage's critic inherits its own generator's trunk config rather
than a new critic_trunk config key, mirroring the existing
critic_hidden_dim/critic_n_res_blocks "0 = inherit from generator" pattern.
Router mixing (MoE) for the critic stays out of scope. Since CriticModel is
training-only and never persisted for inference, and WGAN-GP is still
unbenchmarked, its state_dict shape has no back-compat burden.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Turns "is this component earning its parameters?" into a one-line
config flip for each of the three pluggable network components:
- router.type = "none" (NoneRouter, giant/model/routers.py): still
builds n_experts expert trunks via RoutedTrunk, but replaces the
learned gate with a uniform 1/n_experts weight for every row — no
centers/embeddings/classifier. Distinct from router.enabled=false
(which drops routing/mixing entirely): this isolates whether the
*learned routing signal* specifically is earning its parameters,
holding expert count fixed.
- stage2_model.autoregressive.history = "none" (NoHistory,
giant/model/history.py): ignores feat/has_prev entirely and always
returns zeros, ablating whether the AR decoder's history
conditioning earns its parameters. Already validated for free by
gitea #35's generic HISTORY_REGISTRY membership check.
- trunk.type = "linear" (LinearTrunk, giant/model/trunks.py): a bare
nn.Linear(in_dim + cond_dim, out_dim) body, no ResBlock stack. Per
gitea #33's design, this composes for free with router.enabled=true
("mixture of trivial linear experts").
Both blocking issues (#33 trunk registry, #35 pluggable history
encoder) are closed, so this was unblocked.
The prior e2e run (task 1348) failed on the bump-version job's push step
with a 403 Forbidden — CI_TOKEN lacked write access. Note this on the
checkout step so the requirement isn't lost, now that the token has been
rescoped. Trivial commit to re-open a merge request and re-run the job
end to end.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Version bumps and release tags were entirely manual; the only CI automation
was sync-version-on-tag, which corrects pyproject.toml if a hand-pushed tag
drifted. This flips that: a new bump-version job (needs the four existing
checks, gated to actual merge commits on master via HEAD^@'s parent count so
direct/squash/rebase pushes are untouched) uses bump-my-version to auto-bump
the patch version when a merged branch didn't already bump it itself, then
generates a changelog entry with git-cliff and pushes a matching vX.Y.Z tag.
git-cliff's cliff.toml is tuned to this repo's plain imperative commit style
(no feat:/fix: prefixes): commits are grouped Added/Fixed/Removed/Changed by
leading verb, "(gitea #N)" is linkified, and merge/[skip ci] commits are
dropped. Per user decision during planning: the changelog generator folds in
@lars's comment on the issue (asking to fold in changelog generation rather
than deferring it), and CHANGELOG.md starts fresh with no backfill of
v0.2.0-v0.3.3.
sync-version-on-tag is left untouched as the safety net for hand-tagging.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lars
gitea-actions