V0.3.0 stage2 autoregressive #27

Open

lars wants to merge 30 commits from v0.3.0-stage2-autoregressive into master

pull from: v0.3.0-stage2-autoregressive

merge into: lars:master

lars:master

lars:feat/router-learnable-width

lars:fix/event-id-file-offset

lars:condor-gpu-train-rollout

Conversation 0 Commits 30 Files Changed 86

+15109 -4983

lars commented 2026-08-06 12:26:36 +02:00

Owner

Copy Link

Copy Source

No description provided.

lars added 6 commits 2026-08-06 12:26:37 +02:00

Add v0.3.0 design doc: Stage-2 autoregressive redesign ... f390884f67

Design contract for the v0.3.0 config break and network.py refactor,
following the 2026-08-04 meeting with Jan. Not implemented yet.

The 2026-08-03 WGAN rollout benchmark failed specifically at the
secondary-species level (zero photons, ~4M hallucinated -14 muon
antineutrinos). The response pivots Stage 2 to autoregressive generation
in descending-energy order with teacher forcing, and reverts the particle
type to a categorical representation.

That needs a config break: [conditioning] / [stage1_model] /
[stage2_model] / [train] replace the single global train.mode and
[model] block, so per-stage generators (stage1 flow + stage2 wgan),
stage-2-only training, and one-shot-vs-autoregressive comparison all
become expressible. The particle and material conditioning axes are
configured independently and mix freely, each with physical / embedding /
onehot modes; the stage-2 type target mirrors the same three names, with
conditioning.particle.emb_dim sizing both so the two share one class map.

network.py collapses from ten permutation classes (stage x objective x
routed) into composable parts — encoder x trunk x objective — which also
makes routed WGAN work for the first time; it was only ever rejected
because no routed WGAN generator class existed.

The doc specifies every config option, the v0.2 migration (shim for both
configs and checkpoints, gated on a bit-identical output diff), the
refactor, and the implementation order.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Refine v0.3.0 design: defaults, deferred scope, open questions ... 376bdb9d08

Config defaults: dropout 0.1 -> 0.0, wandb false -> true. The v0.3.0 work
is a sequence of architecture comparisons, and an unlogged run is not
comparable, so W&B is on unless explicitly disabled.

Restructure the open-questions section into settled / deferred / tracked /
still-open, since most of it is now decided:

- other_policy three-way switch and separate flow/ddpm sub-tables are
  approved as specified.
- stop_token is schema-valid but raises "not implemented in v0.3.0";
  charge conservation gets no key at all and is left deliberately
  undesigned, to be worked out on its own terms rather than pre-shaped by
  this refactor. The speculative charge-mask sketch is removed.
- Logging the L1 decode-distance distribution under target = "embedding"
  becomes tracked implementation work, landing with the rollout decode.
- estimate_batch_size recalibration becomes implementation step 8, last:
  the activation-memory profile is not knowable until the AR trunk and
  history encoder are final, so it is measured on real hardware with the
  example configs rather than guessed.

Only the differentiability question for Jan remains genuinely open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Document the differentiability position and its validation obligation ... a489991a3b

The categorical type path is not differentiable, and v0.3.0 accepts that:
the expected contribution of the broken path to the total gradient is
assumed negligible. Record it as an assumption with an explicit obligation
to demonstrate it, not as a settled result.

Separates the three things "broken" covers, since they have different
status: per-token loss under teacher forcing is fine (softmax CE needs no
sampling); ST-Gumbel into the critic is biased rather than absent (hard
forward, soft backward); full shower-rollout backprop was already
structurally non-differentiable once secondaries branch, so the switch
costs nothing that was not already lost. The accepted claim concerns only
the middle one.

Lists three ways to falsify it, cheapest first: gradient-magnitude
accounting through the type slice vs the continuous slices, a
detached-type ablation, and an estimator swap against REINFORCE if those
are inconclusive. The first is wired into implementation step 5 so
evidence accrues during the architecture comparison rather than in a
dedicated run afterwards, and the fallback if the ratio is not small is a
config change (target = "physical" or a non-adversarial CE head), not a
redesign.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

v0.3.0 step 1: new nested config schema, v0.2 migration shim ... eb6dd27406

Replace the single global train.mode + [model] block with the four
top-level blocks docs/v0.3.0-design.md specifies ([conditioning],
[stage1_model], [stage2_model], [train]), so Stage 1 and Stage 2 can
run independent generative objectives and Stage 2 can train standalone.

- migrate_config translates old config.toml/checkpoint dicts on load,
  so nothing on /ceph goes dead; loudly rejects non-zero
  expert_hidden_dim/expert_n_blocks, which v0.3.0 no longer supports.
- merge_cli_overrides/save_config generalize from one hardcoded nesting
  level (model.router) to arbitrary recursive depth.
- default_out_dir_name candidates move to dotted paths against the new
  schema, with per-stage router/generator discriminators.
- validate_config adds cross-block checks the per-block schema can't
  express (particle_type.target=embedding needs a matching conditioning
  mode, tie_to_stage1 needs an active stage 1, etc).
- resolve_expert_dims is deleted (experts always inherit the stage's
  hidden_dim/n_res_blocks now) — pipeline.py/cli.py callers are left
  dangling on purpose, to be updated in the network.py/train.py steps
  that follow.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

v0.3.0 step 2: network.py refactor to composable stage models ... 9ce55e5013

Decomposes the ten permutation classes in giant/model/network.py into
the reusable parts from docs/v0.3.0-design.md §5: ConditionEncoder (now
independently configurable per particle/material axis), ContextAdapter,
Trunk/MonolithicTrunk/RoutedTrunk/ExpertTrunk, and the stage classes
Stage1Model/Stage2OneShot/CriticModel (Stage2Autoregressive stubbed,
raises NotImplementedError until step 4/5). build_models/build_critics
now return a dict keyed by stage and accept the new nested config shape,
with routed WGAN reachable for the first time (the old --mode wgan
--router rejection is gone) and stage2_model.router.tie_to_stage1
sharing a literal Router instance.

A v0.2 checkpoint's flat model_config auto-migrates via
_migrate_legacy_model_config + migrate_legacy_state_dict, preserving the
n_sec_head's attachment to Stage1Model (legacy_owner="stage1", design
doc §4.1). tests/test_migration_v02_v03.py proves this bit-identical
against a frozen v0.2 snapshot (tests/legacy/network_v02_snapshot.py)
for both flow and wgan, both conditioning modes.
scripts/check_migration_v02_v03.py is the real-checkpoint counterpart
for a portal machine with /ceph access.

giant/model/schedule.py's flow-matching/DDPM loss helpers are updated
to the new model-call convention (t as a keyword). giant/sample.py,
giant/rollout.py, and giant/validate.py are not yet updated (deferred
to design doc step 6) — their exercising tests are marked xfail with
that reasoning rather than silently broken.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

v0.3.0 step 3: per-stage train.py trainers + pipeline.py/cli.py rewrite ... 9112e845e0

Replaces train.py's single global training loop with a StageTrainer
hierarchy (FlowDDPMStageTrainer, WGANStageTrainer) — one per active
stage, each owning its own optimizer/LR schedule/EMA and reading only
the shared batch tuple (stage 2 always teacher-forces on the
ground-truth x1_s1, so stages never need each other's output at train
time). Supports every stage1/stage2 generator combination, including
the design doc's headline mixed case (stage1=flow + stage2=wgan) and
its reverse, plus stage1-only/stage2-only ablation runs, routed+gumbel
stages, and checkpoint save/resume. metrics.csv/wandb logging are
stage-prefixed. validate_marginals calls are guarded with a one-time
warning and a Wasserstein-magnitude fallback for wgan best-checkpoint
selection, since giant/sample.py still assumes stage1 always owns
n_sec_head (decision 1 moved it to stage 2 by default) — deferred to
design doc step 6, not silently papered over.

pipeline.py's run_setup_stage/run_train_job now read the new nested
config directly; the dangling resolve_expert_dims call and the
--mode wgan --router rejection are both gone (routed WGAN works).
cli.py's train/new-run build correctly-shaped config overrides
(architecture flags -> stage1_model only per the approved decision;
--mode/--n-critic/--gp-weight/--critic-lr broadcast to both stages,
matching migrate_config's own precedent and avoiding a regression on
the common --mode case); predict/rollout's dangling build_models
tuple-unpack is fixed; new-run now tags config_version, fixing a bug
where a re-loaded v0.3 config.toml would have been silently corrupted
by migrate_config mistaking it for v0.2.

config.py's validate_config rejects mixed particle/material
conditioning types for now (ConditionEncoder supports it, the data
pipeline in giant/data/transforms.py doesn't yet). analysis/render.py
and router_gating.py handle both the new nested model_config shape and
legacy flat checkpoints. scripts/warm_setup_cache.py updated for
run_setup_stage's new signature.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-06 15:54:44 +02:00

v0.3.0 step 4: type map + particle_type.target = "onehot"/"embedding" ... 4fc15ecdfc

Builds the shared top-N-plus-other PDG/material maps (pooling both primary
and secondary occurrences for PDG, directly targeting the meeting's
species-collapse failure mode) and wires up conditioning.{particle,material}
= "onehot" plus stage2_model.particle_type.target in ("onehot", "embedding")
end-to-end: setup-cache persistence, Stage2OneShot's type_head (flow/ddpm)
vs. folded+ST-Gumbel-relaxed adversarial slice (wgan), and the corresponding
CE/MSE training losses. particle_type.target = "physical" stays byte-for-byte
unchanged, keeping the v0.2 migration shim's bit-identical guarantee intact.
giant predict/rollout fail loudly on a onehot/embedding checkpoint until
full decode support lands in step 6.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-07 09:37:19 +02:00

v0.3.0 step 5: Stage2Autoregressive (history=markov) + §11.4 grad instrumentation ... c9d255b1c5

Replaces the Stage2Autoregressive stub with a real per-token secondary
decoder: MarkovHistory summarizes the previous secondary, remaining-energy
fraction and slot index round out the per-token conditioning, and the
existing Trunk/MonolithicTrunk/RoutedTrunk machinery is reused unchanged by
batching all K_MAX tokens together under teacher forcing (one parallel pass,
no new trunk code). build_models/build_critics wire it in; the WGAN critic
stays whole-sequence, so build_critics needs no AR-specific path.

train.py's FlowDDPMStageTrainer/WGANStageTrainer gain a decoder branch,
sharing optimizer/EMA/checkpoint machinery with the one-shot path.
_assemble_stage2_real is now defined in terms of the new unflattened
_assemble_stage2_ar_target helper, removing a near-duplicate branch.

Also lands the §11.4 differentiability validation-obligation instrumentation
(trunk-gradient norm from the particle-type slice vs. the continuous slices,
for generator=wgan + particle_type.target=onehot) via backward hooks in
_relax_onehot_type_slice, decoder-agnostic and surfaced as two new
metrics.csv columns.

This also fixes the standing regression where any config not explicitly
overriding decoder="one_shot" crashed at build_models, since
stage2_model.decoder defaults to "autoregressive" — confirmed by removing
tests/test_pipeline.py's now-stale override so the default config runs
end-to-end against real synthetic data.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-07 10:38:49 +02:00

v0.3.0 step 6: sample.py/rollout.py AR generation + class->PDG decode ... 93b19911f8

- giant/sample.py: fix every sampler's call convention against
  Stage1Model/Stage2OneShot's actual forward signatures (was still
  calling model(x, t, cond_cont, cond_cat) positionally); add
  sample_secondaries_ar (free-running AR loop, unsnapped history feature)
  and sample_stage1/sample_stage2/resolve_n_sec dispatch helpers that read
  each stage's generator_kind/decoder off the model instance itself.
- giant/particles.py: decode_topn_class (argmax + other_policy) and
  decode_embedding_nearest (L1-snap + distance) turn a secondary's
  "onehot"/"embedding" type prediction into a concrete PDG.
- giant/rollout.py: decode_secondary_identity routes all three
  particle_type.target values to real mass/charge; per-stage generator
  dispatch (drops the single shared `mode` string, adds ddpm support);
  L1DistCollector accumulates the §11.3 embedding-distance diagnostic.
- giant/cli.py: drop the onehot/embedding-target rejection gate (narrowed
  to the still-unimplemented conditioning.particle/material.type=onehot
  axis); fix the dead model_cfg.get("mode") bug in predict/rollout.
- giant/analysis/: new type_embedding_l1_distance PlotSpec, wired through
  the rollout YAML sidecar (no live-model call needed, unlike
  router_gating -- the histogram is already pre-aggregated at rollout
  time).
- Un-xfail every test that was blocked on this step (test_rollout.py,
  test_flow.py, test_wgan.py, test_phase2.py, test_router.py,
  test_validate.py); add test_sample.py, test_type_embedding_distance.py.

Known follow-up: giant/validate.py still unpacks the training val-batch
as a stale 6-tuple and doesn't use the new per-stage dispatch, so
marginal validation during training degrades gracefully with a warning
rather than working -- not in this step's scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 2 commits 2026-08-07 16:13:41 +02:00

v0.3.0 step 7: AttentionHistory (KV-cached) + scheduled/never teacher forcing ... 200c6d243b

AttentionHistory (giant/model/network.py) adds causal self-attention over
the emitted-secondary prefix as the alternative to MarkovHistory, with a
parallel forward() for training and an init_cache()/step() KV-cache path
for sample.py's per-slot AR inference loop, wired into
Stage2Autoregressive via history="attention".

giant/train.py adds _stage2_tf_prob and _assemble_stage2_ar_inputs_scheduled,
mixing ground-truth history with a detached sample_secondaries_ar self-sample
per slot so teacher_forcing="scheduled"/"never" close the train/inference gap
teacher_forcing="always" always avoided; wired into both stage-2 AR trainers.

config.py's validate_config no longer rejects these two previously
unimplemented schema values.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

v0.3.0 post-implementation audit: resolve all 9 tracked discrepancies ... da7cde3ef9

Works through docs/v0.3.0-followups.md item by item, closing the gap
between the design doc and the shipped v0.3.0-stage2-autoregressive code:

1. validate.py: 7-tuple batch unpacking, sample_stage1/sample_stage2
   dispatch, stage-2 particle-type-class marginal.
2. Stage-prefixed --stage1-*/--stage2-* CLI flags for train/new-run.
3. Thread stage2_model.k_max through loader/transforms/dataset/pipeline/
   train instead of the hardcoded K_MAX constant.
4. Mixed conditioning.particle.type / conditioning.material.type support
   end-to-end (data pipeline + dwarf warm-cache).
5. conditioning.share_stages = true: one shared ConditionEncoder instance
   across both stages.
6. stage2_model.generator = "ddpm" formally deferred into design doc §11.2
   (was silently unimplemented).
7. giant predict/rollout: implement conditioning.*.type = "onehot" via the
   checkpoint's saved pdg_topn_map/mat_topn_map.
8. network.py's checkpoint-path model_config migration now fails loudly on
   non-zero legacy expert_hidden_dim/expert_n_blocks, matching config.py's
   TOML-load path (§4.2).
9. validate_config now rejects stage2_model.n_sec.mode = "truth" for a
   rollout-capable checkpoint (§9).

Also cleared all pre-existing `ty check` noise (44 -> 0 diagnostics),
mostly a test-helper dict-unpack pattern that made every unrelated
constructor keyword look like a type error.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-07 17:04:32 +02:00

Refactor train.py into giant/training/ around a metrics collector ... 8019a80563

Every metric name used to exist in four places: the dict keys each
StageTrainer returned, the hardcoded _metrics_fields() column list, the
~110-line metrics_row assembly in train(), and the tqdm/summary
formatting. The two had to be kept in exact correspondence by hand or
csv.DictWriter would raise.

Each metric is now declared once, as a MetricSpec on the trainer that
computes it. MetricsCollector derives the CSV header and W&B payload from
those declarations and owns all accumulation, so train() no longer carries
a running sum, and every isinstance(tr, WGANStageTrainer) branch is gone —
replaced by four trainer hooks (batch_loss, summary, val_objective,
supports_val_loss).

giant/train.py (1875 lines) becomes giant/training/:
  trainers.py       StageSpec + shared StageTrainer base + the two subclasses
  metrics.py        MetricSpec, MetricsCollector
  stage2_inputs.py  the pure AR/teacher-forcing tensor helpers, moved verbatim
  loop.py           train() (225 lines, was ~514) + graceful shutdown
  checkpoint.py     build/load, lifted out of train()'s closures

The trainers shared ~15 identical constructor arguments and copy-pasted
their cosine-warmup lambda, EMA setup, state_dict/load_state_dict,
resume_lr and train_mode/eval_mode. StageSpec resolves one stage's config
once (constructors go from 24 and 22 keyword arguments to (spec, model,
device)), the base class holds the rest, and build_stage_trainers drops
from ~100 lines to 15.

Metric columns are renamed to a uniform stage/split/metric scheme
(stage1/train/loss, stage2/train/d_loss, stage1/lr, stage1/router/entropy,
val/loss, ...). Old metrics.csv files and W&B history are not comparable.
The checkpoint format is unchanged.

BEHAVIOR CHANGE — WGAN best-checkpoint selection. The old code meant to
score a WGAN stage on its marginal KL, but the guard
`{n: kl for n in wgan_names if n not in val_loss_per_stage}` could never
fire: val_loss_per_stage was pre-seeded with 0.0 for every stage, so a
WGAN stage contributed a flat 0.0 and the KL was written to metrics.csv
without ever influencing best.pt. val_objective now returns it as
intended. On the test harness's default flow+wgan config val_loss went
from 2.182 (stage 1 only) to 15.137 (stage 1 + KL 12.954), and which epoch
won changed. Runs before this commit picked their best checkpoint on the
non-adversarial stages alone. Written up in docs/v0.3.0-followups.md.

Verified: 699 tests pass; ruff, ruff format and ty clean. Baseline-vs-
refactor metrics.csv compared across five configs (flow+wgan, AR+onehot,
routed, both-flow, AR-flow) — every comparable value bit-identical except
val/loss where the fix applies. Resume appends without a duplicate header
and reproduces a HEAD worktree's per-epoch losses and LRs exactly across
the resume boundary. A refactored last.pt loads through
cli.py:_load_model_weights in both raw and ema modes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 09:49:42 +02:00

Silence the fork-safety warning from num_workers>0 pipeline tests ... d3271bc798

The two DataLoader-num_workers quota tests are the only ones in the file
that leave num_workers>0, so they're the only ones that actually spawn
forked worker subprocesses under pytest's multi-threaded process and hit
Python's fork-safety DeprecationWarning. The thing under test is just the
pre-flight quota-check message, emitted before the DataLoader is built.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 10:32:32 +02:00

Deduplicate giant/training/trainers.py shared per-stage logic ... fff61ebd61

Lift repeated per-batch operations into StageTrainer base-class helpers so
each is written once instead of being copy-pasted between FlowDDPMStageTrainer
and WGANStageTrainer:

- _n_sec_loss: the multiplicity classifier (stage1/stage2 predict_n_sec split
  + cross-entropy + accuracy), previously written three times. Gated on
  n_sec_head presence, not n_sec.mode, so a future stop_token model trains its
  EOS signal elsewhere and this stays zero.
- _sec_mask: the arange < n_sec prefix mask, previously in two places.
- _step_optimizer: the zero_grad/backward/clip_grad_norm_(1.0)/step quad,
  previously written three times; now the single home of the clip constant.
- _sec_target: collapses the byte-identical _ar_target/_real wrappers into one
  flatten-parameterized method (they differed only by .flatten(1)).

Also trim StageSpec.from_config to read DEFAULT_CONFIG-guaranteed train.* keys
directly instead of re-defaulting them.

The three particle-type targets (onehot CE, physical/embedding regression) and
_type_loss are intentionally left as separate paths — genuinely different
objectives, not duplication.

stage2_inputs.py: extract the shared _ar_meta helper for the has_prev/
remaining_frac/slot_idx trio used by both AR-input assemblers.

Behavior-preserving: same losses, optimizer order, and RNG draw order. Full
test suite (699) green; ruff + ty clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 10:49:55 +02:00

Rewrite README for v0.3.0 architecture, quick start, and data columns ... 630d8d3992

The two-stage architecture description had drifted from the v0.3.0
stage-2-autoregressive redesign (8 commits, eb6dd27..da7cde3) — it still
documented the old one-shot-only SecondaryDecoder and continuous
mass/charge secondary target. Restructured for faster onboarding: a
Quick start section up front, bullet-point Architecture and training-flag
docs instead of dense paragraphs, and a Data section listing the actual
parquet columns consumed by giant/data/loader.py. Dropped the Roadmap
section (status/history, not architecture) and CLI-flag default callouts
from Architecture, keeping it focused on net structure.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 2 commits 2026-08-10 11:21:17 +02:00

Bump version to 0.3.0 ... f46628141d

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Delete docs/v0.3.0-design.md and strip all references to it ... 878e9ddca3

The design doc and its followups doc are no longer needed as a live
reference now that the v0.3.0 redesign is implemented — comments and
docstrings across the codebase cited it extensively (file path, "design
doc §X.Y", "decision N", or bare "§X.Y" section numbers) as design
rationale. Removed docs/ and edited every citing comment/docstring to
drop the now-dangling reference while keeping the substantive
explanation next to it. CLAUDE.md's v0.3.0 roadmap bullet loses its
trailing pointer to the deleted file.

Verified: no remaining "docs/v0.3.0", "design doc", "decision N", or
"§N.N" references (repo-wide grep); ruff and ty clean; full test suite
on the heaviest-touched modules (network, sample, rollout, migration,
config, train) passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 11:25:56 +02:00

Add pytest-cov to dev deps and run coverage in CI ... adb7a8663e

Test job now reports coverage (term + xml) and uploads it as a build
artifact, so coverage regressions are visible per-PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 11:26:31 +02:00

Apply ruff format ... 451bdc210e

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 11:38:09 +02:00

Add coverage for router-center seeding, geometry batch reader, material topN cache, and setup-cache corruption paths ... 24445b7427

Closes the highest-value coverage gaps found via pytest-cov: pipeline.py's
EnergyRouter quantile-seeding (the roadmap's flagged fix for the failed MoE
rollout benchmark) had zero coverage, geometry.py's real parquet-batch reader
was always mocked, the material top-N-map cache-hit branch was untested
(only pdg's was), and setup_cache.py was missing malformed-cache-body and
unknown-axis error paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 11:38:58 +02:00

Downgrade coverage-report upload to actions/upload-artifact@v3 ... b1cf9d345d

v4 requires the @actions/artifact v2 backend, which this self-hosted Gitea
instance doesn't support yet (GHESNotSupportedError) — v3 uses the older
API Gitea's Actions runner implements.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 11:47:33 +02:00

Add render.py coverage: figure params, router diagnostics plots, gallery/condor glue ... 82772e4e09

render.py was at 58% coverage — the module's plotting dispatch (router_gating,
router_share, unavailable) and glue logic (_figure_params/_figure_params_v2,
_plot_metadata, render_all's gallery subprocess call, render_run's condor
RunMeta wiring) had no tests at all. Brings it to 100%: pure-function unit
tests for the v0.2/v0.3.0 figure-param branches and _plot_metadata, real
LaTeX-rendered fixtures for the previously-untested plot kinds and a
4-group grouped_hist (exercises the hidden-leftover-axis branch), and
mocked subprocess/condor calls to isolate render_all/render_run's own logic.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-10 11:56:39 +02:00

Fix test_render_all_run_gallery_invokes_subprocess clobbering LaTeX's own subprocess.run ... 9ce7b32324

render_mod.subprocess is the stdlib subprocess module itself, not a copy —
patching .run unconditionally also intercepted the real subprocess.run
calls matplotlib's texmanager makes to compile LaTeX during savefig, so
those returned the test's fake return value instead of a real
CompletedProcess and crashed with AttributeError: 'NoneType' object has no
attribute 'stdout' on any environment where render_all runs before the
gallery call (i.e. everywhere but this dev machine's warm state that
happened to mask it). Only intercept the "gallery generate" call now;
everything else passes through to the real subprocess.run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars marked the pull request as ready for review 2026-08-10 12:06:45 +02:00

lars added 1 commit 2026-08-12 13:33:23 +02:00

Bump ruff line-length to 120 and reformat ... 55332db67a

Rejoins lines that only wrapped because they exceeded the old 88-char
limit; ruff check and the full test suite (725 passed) are unaffected.

lars added 1 commit 2026-08-12 14:36:03 +02:00

Make config dataclasses the single source of truth for DEFAULT_CONFIG ... 9bf5874308

DEFAULT_CONFIG and build_models/build_critics/StageSpec.from_config's
inline .get(key, default) fallbacks had already drifted: two keys
(stage2_model.decoder, stage2_model.particle_type.target) resolved
differently depending on whether a config dict came from
merge_cli_overrides (fully populated, correct) or was hand-built and
partial (fell back to stale v0.2-shaped literals). Introduce frozen
dataclasses (GiantConfig and its nested blocks) in giant/config.py as
the actual single declaration of every default; DEFAULT_CONFIG is now
generated from them instead of hand-maintained, and build_models,
build_critics, and StageSpec.from_config consume the dataclasses
instead of duplicating literal fallbacks, so this class of drift can't
recur. Router/n_sec sub-blocks keep an `extra` catch-all for their
genuinely dynamic keys (composed-router axes, runtime-seeded
centers_init, legacy_owner).

Fixing the fallback surfaced the same latent bug in two existing
partial-config callers that had been silently depending on it: a
test fixture in test_train.py and scripts/warm_setup_cache.py's
minimal cfg (now merged against DEFAULT_CONFIG instead of hand-rolled,
closing the gap for good). See issues.md Issue 1.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 1 commit 2026-08-12 14:48:01 +02:00

Add unknown-key validation to config.toml merge (issues.md Issue 2) ... 01acbfed61

A typo like `n_res_block` for `n_res_blocks` previously merged cleanly,
passed validate_config, and silently trained a model that didn't match
config.toml's documented settings. merge_cli_overrides now rejects any
key not present in DEFAULT_CONFIG's schema via validate_config_keys,
with a did-you-mean suggestion, while still allowing the genuinely
dynamic composed-router axis keys and centers_init. Checkpoint
model_config loading is untouched, so old checkpoints keep loading.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 2 commits 2026-08-12 15:06:10 +02:00

Extract giant train/new-run's CLI override mapping into a table-driven function (issues.md Issues 3 & 4) ... 2bfb1ab056

train()'s ~140-line hand-written flag->config translation (three different
ad hoc "more specific flag wins" patterns) and new_run()'s near-verbatim
copy are replaced by a shared FlagSpec/FLAG_SPECS table and
overrides_from_flags() in config.py, reused by both commands. This makes
the override/precedence logic directly unit-testable without CliRunner,
closing coverage gaps that had zero tests (e.g. --emb-dim/--conditioning
dual-axis fan-out, three of four WGAN knob legs, --stage2-generator
overriding --mode, router's stage1-only asymmetry).

No CLI flags, help text, or precedence semantics changed --
`giant train --help`/`giant new-run --help` are byte-identical before and
after, and all previously-passing CliRunner tests still pass unmodified.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Mark issues.md Issues 3 & 4 as fixed ... 6a21c3b908

Records what commit 2bfb1ab actually changed and its scope, matching the
status-blockquote convention already used for Issues 1 and 2.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

lars added 2 commits 2026-08-12 15:31:24 +02:00

Extract predict/rollout's duplicated inference bootstrap into giant.checkpoint_io (issues.md Issue 5) ... 818c380fd0

giant predict and giant rollout each carried a ~65-line, independently
drifting copy of "load checkpoint -> validate -> resolve conditioning axes
-> restore normalizers/vocab maps -> build models -> load weights", plus a
third partial copy of _conditioning_axes in analysis/router_gating.py. A
silent divergence there doesn't crash, it makes the two commands run
different physics from the same checkpoint with no test coverage anywhere
along that path.

giant/checkpoint_io.py now holds the single implementation:
load_for_inference() + an InferenceContext dataclass, raising
CheckpointCompatibilityError (verbatim message text preserved) instead of
calling typer directly, so it can be unit-tested and imported from
non-Typer code. router_gating.py's load_router imports conditioning_axes
from it lazily, keeping its "no torch at module scope" contract intact.

Adds 17 direct unit tests for load_for_inference/conditioning_axes/stage_cfg
plus CLI smoke tests confirming the error surfaces as typer.Exit(1) through
predict and rollout — previously zero coverage on this path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Mark issues.md Issue 5 as fixed ... 733c13c31c

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Some checks are pending

Hide all checks

CI / Lint (ruff check) (push) Successful in 27s

Details

CI / Format (ruff format) (push) Successful in 29s

Details

CI / Sync project version with tag (push) Has been skipped

Details

CI / Lint (ruff check) (pull_request) Successful in 45s

Details

CI / Type check (ty) (push) Successful in 48s

Details

CI / Format (ruff format) (pull_request) Successful in 34s

Details

CI / Sync project version with tag (pull_request) Has been skipped

Details

CI / Type check (ty) (pull_request) Successful in 34s

Details

CI / Tests (pull_request) Successful in 3m47s

Details

CI / Tests (push) Successful in 3m54s

Details

This pull request can be merged automatically.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin v0.3.0-stage2-autoregressive:v0.3.0-stage2-autoregressive

git checkout v0.3.0-stage2-autoregressive

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

architecture

experiment

investigation

performance

phase2

validation

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

lars (Lars Bogner)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: lars/giant#27