277 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
lars	f60af64d00	Merge pull request 'Add bf16 autocast to the training loop (gitea #47)' (#68) from fix/issue-47 into master ... Reviewed-on: #68	2026-08-17 15:45:51 +02:00
lars	78978769f6	Add bf16 autocast to the training loop (gitea #47) ... giant/ had no autocast/GradScaler/torch.compile anywhere despite the project's ~10x-native-Geant4 eval-budget target. This adds bf16 mixed precision to the training step (both FlowDDPMStageTrainer and WGANStageTrainer) via a new train.precision config key ("fp32" default, "bf16" opt-in) and giant.training.amp.resolve_autocast. torch.compile is a separate, much larger surface (data-dependent routed dispatch, the autoregressive sampler's per-token control flow, arbitrary rollout batch sizes) and is left for a follow-up issue, per discussion. Scope decisions made during planning: - fp32 + bf16 only, no fp16/GradScaler. fp16 breaks two things in this codebase: routers.py's three 1e-8 epsilons sit below fp16's ~6e-8 subnormal floor, and gradient_penalty's grad norm overflows fp16's range at ordinary early-WGAN-GP gradient magnitudes. Every training GPU in the fleet (A100/L40S/H200/RTX 4070) has native bf16; only pre-Ampere V100s would need fp16. - resolve_autocast raises loudly if bf16 is requested on hardware that can't do it, rather than silently falling back to fp32. - Autocast wraps the training step only; val_loss (and the best-checkpoint selection it drives) stays fp32 so it's comparable across every run recorded so far. - _route_forward's mixture accumulator (giant/model/trunks.py) was a hard-fp32 torch.zeros with no dtype, so under autocast a RoutedTrunk silently returned a different output dtype than an unrouted ExpertTrunk purely because router.enabled was set. Fixed to match the experts' own dtype; the gate weights (forced fp32 for their own numerical stability) are cast down before combining, so the mixture's numerics stay solid without reintroducing the dtype split. - Added explicit fp32 guards (autocast(enabled=False)) around spots that are correct in fp32 but degrade quietly rather than crash in bf16: the router's balance/entropy losses and gate softmax, the stage-2 stick-breaking cumprod, and gradient_penalty's double-backward + grad norm. Benchmarked on the local RTX 4070 against configs/baseline.toml's hyperparams (hidden_dim 512/6 blocks, bs 4096) on a synthetic dataset: bf16 gave 1.05-1.35x training throughput and 18-33% lower peak GPU memory across one-shot/routed/autoregressive stage-2 configs, with the autoregressive path (the dominant cost per baseline.toml) benefiting most on both axes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 15:36:12 +02:00
lars	692acd77eb	Merge pull request 'Add per-stage init_from/freeze (gitea #42)' (#67) from fix/issue-42 into master ... Reviewed-on: #67	2026-08-17 14:48:38 +02:00
lars	e8842c56d7	Bump patch version to 0.3.3 ... Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 14:25:51 +02:00
lars	87e37ebe14	Add per-stage init_from/freeze (gitea #42) ... stage{1,2}_model.active = false already trains one stage alone, but the checkpoint it writes holds only that stage, so giant rollout refuses it -- the "retrain stage 2 alone against a fixed, known-good stage 1" experiment the 2026-08-03 species failure calls for wasn't runnable end to end. Adds stage{1,2}_model.init_from (a checkpoint .pt to load this stage's weights from before training) and .freeze (never update them), symmetric across both stages. Both stages stay active = true, so both get built and both land in the output checkpoint -- the frozen stage is merely initialized from disk instead of from scratch. Decisions made during planning: - Soft freeze: forward/backward still run every batch (loss/grad_norm stay meaningful, no autograd special-casing), only optimizer.step() (and, for the frozen stage, lr_sched.step()/EMA update) is skipped -- weights are byte-identical for the whole run. This is StageTrainer._step_optimizer, shared by the non-adversarial path and both halves (generator + critic) of the WGAN path, so a frozen WGAN stage's critic freezes too. - validate_config requires init_from whenever freeze = true, unless the run is a --resume (a resumed frozen stage's weights come from the resume checkpoint instead) -- freezing a randomly-initialized model is almost certainly a mistake. - CLI flags on both `giant train` and `giant new-run` (--stage{1,2}-init-from/--stage{1,2}-freeze), matching every other per-stage model knob's existing treatment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 14:23:20 +02:00
lars	8290e350b8	Merge pull request 'Implement stage2_model.stage1_context = "sampled" (gitea #41)' (#66) from fix/issue-41 into master ... Reviewed-on: #66	2026-08-17 13:40:54 +02:00
lars	48faaee79d	Implement stage2_model.stage1_context = "sampled" (gitea #41) ... Stage 2 was trained on ground-truth stage-1 outcomes but deployed on sampled ones, and in a rollout that gap compounds over every step of every track — the same train/inference gap teacher_forcing="scheduled" already closes within stage 2, just never applied at the stage boundary. "sampled" was declared in the schema but rejected loudly by validate_config as unimplemented; this lands the real implementation. Mirrors the existing scheduled-sampling precedent rather than a hard switch: new stage2_model.ctx_p_start/ctx_p_end (defaults 1.0 -> 0.0) linearly ramp P(condition on ground truth) from epoch 0 to the final epoch, so stage 2 doesn't chase a wildly moving stage-1 target early in training. Per the plan discussed with the user: the sample is drawn from stage 1's sampling_model() (EMA weights when present, matching what inference actually deploys), mixed per example via a Bernoulli draw (never blended within a row), and validation always uses the ground truth regardless of the schedule. Fixes a latent bug the same pattern would otherwise have hit: every sampler in giant/sample.py flips its model to .eval() with no restore, so sampling from the raw (non-EMA) stage-1 model mid-step now explicitly restores its .training flag afterward to avoid silently corrupting stage 1's own training mode for the rest of the epoch. validate_config now enforces stage1_context in {"truth", "sampled"}, requires both stages active for "sampled" (nothing to sample from otherwise), range-checks ctx_p_start/ctx_p_end, and rejects the ctx_p_start = ctx_p_end = 1.0 configuration as an unadvertised no-op identical to "truth". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 12:27:13 +02:00
lars	09bea2cbff	Merge pull request 'Add giant model summary command (gitea #46)' (#65) from fix/issue-46 into master ... Reviewed-on: #65	2026-08-17 12:08:59 +02:00
lars	cc9646f279	Add giant model summary command (gitea #46) ... giant model summary --config config.toml builds the resolved Stage1/Stage2 graph from a config with no dataset attached (pdg_vocab/mat_vocab are supplied as placeholders via --pdg-vocab/--mat-vocab, since the real training vocab is dataset-derived) and prints per-module parameter counts, trunk in/out widths, which heads exist, and which conditioning/stage1_model/stage2_model config keys actually shaped the build. The consumed-keys half uses differential probing rather than static identifier matching: build once for a fingerprint (submodule presence, every parameter's/buffer's shape+dtype, every plain scalar attribute a module stores on itself), then perturb one leaf at a time, rebuild, and compare. A changed fingerprint (or a raise) means the key is consumed; no change means it's inert *under this particular config* -- e.g. any stage1_model.router.* key when router.enabled=false. A curated _NOT_BUILD_TIME table separates keys legitimately owned by the trainer/sampler/rollout (loss weights, WGAN-GP hyperparameters, teacher-forcing schedules) from genuinely-inert ones, verified against those call sites. A few config keys branch on equality against one specific string literal (n_sec.owner=="stage1", n_sec.mode=="stop_token", particle_type.target=="physical"); a single generic sentinel probe missed all three since the config's current value and the sentinel landed in the same branch, so those three leaves get their real alternative value tried too (_STRING_ALTERNATIVES). giant.config.leaf_paths is promoted out of tests/test_config_consumed_keys.py (previously a private test-local duplicate) so both audits -- the static per-identifier one and this new runtime per-config one -- walk the exact same DEFAULT_CONFIG tree. ExpertTrunk/RoutedTrunk now also expose in_dim (out_dim already existed), needed to report trunk widths generically. Decisions made during planning: --pdg-vocab/--mat-vocab default to 300 and len(MATERIAL_PROPERTIES); the consumed-keys report is scoped to conditioning/stage1_model/stage2_model only (train/meta are out of scope for a model-only build); the module tree prints every submodule at any depth. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 11:48:02 +02:00
lars	59eccbb5cb	Merge pull request 'Fix/issue 40' (#64) from fix/issue-40 into master ... Reviewed-on: #64 v0.3.2	2026-08-17 10:55:33 +02:00
lars	b42fa95d1a	Bump patch version to 0.3.2 ... Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 10:45:15 +02:00
lars	c1c4957e2f	Implement n_sec.mode = "stop_token" for the AR secondary decoder (gitea #40) ... Stage 2's autoregressive decoder still predicted multiplicity the v0.2 way: a one-shot n_sec_head classifier over conditioning alone, run before any secondary token existed, with the AR loop then always executing k_max slots and discarding the tail. This adds a real per-slot EOS mechanism instead: - Stage2Autoregressive gains a stop_head (build_stop_head=True) that predicts P(n_sec == k | prefix) at each slot, mutually exclusive with n_sec_head (n_sec.mode = "stop_token" builds no n_sec_head at all). - sample_secondaries_ar accepts n_sec_pred=None to drive generation off the stop head instead of a pre-resolved count: each row stops the first slot its stop logit fires (stage2_model.n_sec.stop_sampling = "greedy" — the default, threshold at 0 — or "sample", a Bernoulli draw), and the whole batch loop breaks once every row has stopped, so cost scales with the realized n_sec instead of a fixed k_max. Passing n_sec_pred explicitly (the scheduled-sampling self-sample path) is unchanged. - resolve_n_sec returns None for a stop-token decoder instead of raising; rollout.py/cli.py/validate.py now derive the realized count from sample_stage2's returned sec_valid (sec_valid.sum(-1)) after sampling, rather than resolving it up front — a no-op reordering under every other n_sec.mode, where sec_valid was already built from n_sec_pred. - Training: _stop_target_and_mask (giant/training/stage2_inputs.py) builds the per-slot target/mask (one slot wider than the existing token-content sec_mask, since the stop slot itself needs supervision) and StageTrainer._stop_loss trains it with masked BCE, gated on stop_head exactly like _n_sec_loss gates on n_sec_head. Wired into both the flow/ddpm trainer and the WGAN trainer (whose skip_g_step now also checks stop_head), weighted by the existing stage2_model.n_sec.lambda — the stop head replaces n_sec_head under this mode, so no new weight key. - validate_config now accepts stop_token (requires decoder="autoregressive" and n_sec.owner="stage2") instead of always rejecting it. Decisions made during planning: stop_sampling defaults to "greedy" for deterministic rollouts; the stop head reuses stage2_model.heads.n_sec's HeadConfig shape and stage2_model.n_sec.lambda's weight rather than adding new config keys, since the two heads never coexist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 10:44:14 +02:00
lars	7bf0bea56a	Merge pull request 'Clamp analysis histogram bins before the i32 cast, not after (gitea #61)' (#63) from fix/issue-61 into master ... Reviewed-on: #63	2026-08-17 10:17:32 +02:00
lars	867a07da2b	Merge branch 'master' into fix/issue-61	2026-08-17 09:51:37 +02:00
lars	7514a4364f	Merge pull request 'Clip raw predicted log_mass in decode_secondaries (gitea #54)' (#62) from fix/issue-54 into master ... Reviewed-on: #62	2026-08-17 09:42:52 +02:00
lars	a746efb6e1	Clamp analysis histogram bins before the i32 cast, not after (gitea #61) ... _bin_expr in giant/analysis/reduce.py clipped the bin index to [0, nbins-1] only after casting it to Int32, so the clip never got a chance to run: a rollout step_length of 1.0725e10 mm against fixed edges [2.9e-5, 94.04] with 50 bins produces a raw index of ~5.7e9, which overflows i32 and fails the strict cast, killing the whole compute-one job. Same failure mode for +/-inf. Clamp in f64 first, then cast to Int32. NaN has no edge to clamp to, so it maps to null and is dropped in the two callers (hist1d, profile_partial) — matching what np.histogram does with NaN, and what profile_partial needs anyway since a null bin index would break its np.add.at. This reimplements commit 313373c, which fixed the same bug but landed on a branch (fix/rollout-negative-secondary-mass) that forked off a stale master and was never merged; reduce.py has since diverged enough that the original diff no longer applies cleanly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 09:40:09 +02:00
lars	bacc8763d0	Clip raw predicted log_mass in decode_secondaries (gitea #54) ... decode_secondaries inverted a secondary's raw predicted log_mass with inv_log_transform (exp(y) - eps) unclipped. log_mass is a raw regression output, not itself the result of log_transform, so it isn't guaranteed to land in the range that round-trips cleanly: too negative and exp(y) undershoots eps, making the result go slightly negative; too positive and exp(y) overflows float32 to inf. Either one crashes the next rollout step, since a track descended from that secondary feeds its mass back in as conditioning, and log_transform raises on a non-finite input. Clip log_mass to [log(_EPS), _LOG_MASS_MAX] before inverting, guaranteeing a finite, non-negative mass. _LOG_MASS_MAX=80.0 matches the value from the stale fix/rollout-negative-secondary-mass branch (comfortably below float32's ~88.7 overflow point, far beyond any physical particle mass a converged model would predict) — that branch had already implemented this fix but forked before gitea #35/#36 and couldn't be merged as-is, so this reimplements it fresh against current master and leaves the stale branch untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-17 09:31:37 +02:00
lars	ff435883ed	Merge pull request 'Let dwarf warm-cache take --config so it can't under-warm a config's cache keys (gitea #59)' (#60) from fix/issue-59 into master ... Reviewed-on: #60	2026-08-17 09:24:42 +02:00
lars	d25dfc0343	Let dwarf warm-cache take --config so it can't under-warm a config's cache keys (gitea #59) ... warm-cache built its config from DEFAULT_CONFIG with only a handful of flags overridable, so it had no way to express settings like stage2_model.particle_type.n_classes. configs/baseline.toml sets that to 32; warm-cache always warmed the pdg top-N map under the emb_dim default (16) instead, so a `giant train --config configs/baseline.toml` run silently missed the cache and repaid the full parquet scan warm-cache exists to avoid. warm-cache now accepts the same --config a training run takes and resolves every value run_setup_stage needs (val_fraction/seed, conditioning types, both stages' router, particle_type.n_classes, ...) from one gconfig.merge_cli_overrides + validate_config pass, exactly like giant train's own pipeline does — so warming and training are guaranteed to agree. Per user decision, --config is mutually exclusive with the individual --val-fraction/--seed/--particle-conditioning/ --material-conditioning/--router*/flags (rejected outright rather than silently layered on top), since a hardcoded CLI default clobbering an unset config value is the same failure mode one level down. Also drops a hardcoded stage2_model.router/k_max override that was a no-op against today's defaults but would have clobbered a config setting either one away from its default — same bug class. Adding validate_config surfaced that the existing test_warm_cache_router_process_warms_proc_map test was warming a router.type="process" + conditioning.particle.type="physical" (the CLI's old hardcoded default) combination that giant train's own validate_config would already reject as incompatible — fixed by passing --particle-conditioning embedding, which is what a working --router-type process run actually requires. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-17 08:54:14 +02:00
lars	a1ecf0df1d	Merge pull request 'Add configs/baseline.toml as the kept reference model' (#58) from add/baseline-config into master ... Reviewed-on: #58	2026-08-14 17:37:46 +02:00
lars	d858226294	Add configs/baseline.toml as the kept reference model ... A fixed comparison point for future architecture variants, so each experimental axis (routed trunk, WGAN generators, attention history, shared conditioning) is a single edit away from one known config. flow/flow autoregressive, hidden_dim 512 / 6 blocks per stage, physical conditioning, no router, 7.70M params. Chosen by ranking the five runs in analysis_runs/ by mean Jensen-Shannon divergence against the Geant4 reference: unrouted flow wins (0.172) over routed flow (0.197/0.200) and both WGAN runs (0.218/0.234), with the lead concentrated in per-event total deposited energy and the per-PDG marginals. batch_size 36864 is sized for one L40S on deepthought2 from a measured linear fit of this config's training step (reserved MiB = 0.9736 * bs + 115), giving ~36 GiB, 78% of the card. The comments record two measured facts that are easy to get wrong: WGAN is slower to *train* than flow (n_critic plus the gradient-penalty double-backward), its advantage being inference-only; and sample_secondaries_ar loops over all k_max slots unconditionally rather than short-circuiting on n_sec, which is what makes the autoregressive decoder the dominant cost on both axes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 17:36:26 +02:00
lars	4f092c4528	Merge pull request 'Give Stage1Model/Stage2OneShot/Stage2Autoregressive a shared StageModel base (gitea #39)' (#56) from fix/issue-39 into master ... Reviewed-on: #56 v0.3.1	2026-08-14 15:16:02 +02:00
lars	cc37a55183	Bump patch version to 0.3.1 ... Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 15:05:32 +02:00
lars	c4b12b5e7a	Pass ConditioningAxisConfig/ParticleTypeConfig themselves instead of raw dicts (gitea #38) ... build_models/build_critics parsed model_config into frozen dataclasses (ConditioningConfig, Stage2ModelConfig, ...) but then threw the parsed sub-objects away and passed the original raw dicts (conditioning["particle"], s2_spec.particle_type.to_dict()) down into ConditionEncoder/StageModel/etc, which re-read them with their own hardcoded .get(key, default) fallbacks — each an independent copy of a fact the dataclass already stated once. Worst instance: giant/training/trainers.py:236 converted an already-parsed ParticleTypeConfig back into a dict for no reason. Threads ConditioningAxisConfig (particle_cfg/material_cfg) and ParticleTypeConfig (particle_type_cfg) as the actual dataclass instances through every signature that used to type them dict: ConditionEncoder, StageModel/CriticModel, resolve_type_n_classes/stage2_type_dim/ stage2_trunk_sec_dim, giant/model/builders.py, giant/sample.py, giant/training/stage2_inputs.py, giant/training/trainers.py (StageSpec/ StageTrainer), giant/pipeline.py, giant/rollout.py, giant/validate.py — so ty now catches a misspelled field instead of it silently falling back. No config-schema change: config.toml/checkpoint model_config keep the same nested-dict shape; only what happens after the existing X.from_dict(...) parse changes. User-confirmed scope decision: both axes (particle_cfg/material_cfg and particle_type_cfg), not just the more heavily-duplicated particle_type_cfg axis, and not stopping at the two most literal parse-then-discard round trips — matching the issue's own proposal. Preserved-default decision: StageModel's particle_type_cfg=None sentinel (hit only by direct/test construction — build_models always passes an explicit particle_type) still resolves to ParticleTypeConfig(target= "physical"), not ParticleTypeConfig()'s own target="onehot" config-file default — switching it would have silently grown an unused, gradient-less type_head on every test that constructs Stage2OneShot/Stage2Autoregressive without particle_type_cfg=, breaking their "every param has a grad" checks. New tests in tests/test_network.py: ConditionEncoder/StageModel store the exact ConditioningAxisConfig/ParticleTypeConfig instance passed in (identity, not just equality) — no internal dict round-trip — and build_models's output carries real dataclass instances end to end, not the plain dicts it produced before this fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 15:03:55 +02:00
lars	1a3c907571	Give Stage1Model/Stage2OneShot/Stage2Autoregressive a shared StageModel base (gitea #39) ... Stage1Model, Stage2OneShot and Stage2Autoregressive each independently implemented ~90 near-identical lines of __init__ scaffolding: build-or-share cond_enc, particle_type_cfg normalisation, objective -> time_emb -> merged_cond_dim -> build_trunk, and the n_sec_head/type_head classifier heads (plus their identical RuntimeError guards). Now unblocked by #33 (trunk registry), #34 (block-conditioning registry) and #36 (build_mlp_head), which settled what belongs in the shared base. Adds StageModel(nn.Module) owning all of that: __init__ builds/shares cond_enc and normalises particle_type_cfg; _build_trunk_and_heads, called by each subclass after it sets up its own conditioning-assembly modules (cond_enc alone for Stage1Model, a context-fusion path for the two Stage2 classes), builds the objective/time embedding/trunk and the n_sec_head/type_head guarded by the shared _require_n_sec_head/ _require_type_head (Stage1Model overrides the n_sec guard since its message points at stage 2, not stage 1). Public __init__ signatures, attribute names, and forward/predict_* behaviour are unchanged. Verified with a pre/post state_dict-key-set diff against the pre-refactor classes (bit-identical) before writing this commit, plus new parametrized tests pinning each class's state_dict key set and the generator -> time_emb contract the base now owns. tests/test_migration_ v02_v03.py's existing bit-identical old-vs-new forward comparison and the rest of tests/test_network.py's per-class coverage pass unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 14:37:58 +02:00
lars	c71210f006	Merge pull request 'Give the cond_cat/cond_cont column layout one owner (gitea #37)' (#55) from fix/issue-37 into master ... Reviewed-on: #55	2026-08-14 14:24:48 +02:00
lars	4692cee699	Give the cond_cat/cond_cont column layout one owner (gitea #37) ... The conditioning arrays' column order was written down three times — twice in giant/data/transforms.py (build_cond_features and build_features each built cond_cont and cond_cat from scratch) and again in giant/model/encoders.py (cat_col_layout, plus hand-written COND_DIM_BASE + PARTICLE_PHYS_DIM slicing in ConditionEncoder). The three were held in sync only by parallel comments, so a wrong column order produced silently mis-indexed features rather than an exception. The drift had already happened, twice, both times in build_features: - 5b63dfd added per-axis vocab-lookup strictness (an out-of-vocab pdg/material must not KeyError under "physical"/"onehot", where the index is never read) to build_cond_features only. - _cond_normalizer_transform's legacy-normalizer padding, which keeps a pre-physical-conditioning 8-wide cond normalizer loadable, was likewise only wired into build_cond_features — so `giant predict` on such a checkpoint died with a broadcast error. New giant/cond_layout.py holds a frozen CondLayout built from the (particle, material) mode pair, exposing named cond_cont slices (base/particle_phys/material_phys) and cond_cat columns (PDG_COL/MAT_COL/particle_topn_col/material_topn_col/cat_dim). Both builders now share one _build_cond_arrays, ConditionEncoder reads its slices off the same object, and PdgRouter/ProcessRouter use the named dense-vocab columns instead of literal 0/1. CondLayout also absorbs the two duplicated axis-type validations, keeping their message text verbatim. Decisions taken while planning: - Scope is CondLayout only. The issue's second half — a CONDITIONING_AXIS_REGISTRY registering (feature_columns, encoder_module) as a pair — is deferred: it would force ConditioningConfig's fixed particle/material fields into a dynamic axis map and ripple through pipeline.py, checkpoint_io.py and rollout.py, i.e. a config-schema break with no consumer yet. - The two divergences above are unified onto build_cond_features' behaviour rather than preserved as parameters, so the new single source of truth doesn't carry the old split forward. Each gets a regression test that fails before this commit. - cat_col_layout is replaced outright (deleted, dropped from network.py's __all__, its four tests rewritten against CondLayout) rather than kept as a wrapper — two spellings of the same fact is the defect itself. cond_cat's width is now the layout's call rather than "did the caller pass a map", so an "onehot" axis without its top-N map raises instead of yielding a narrower array that ConditionEncoder would index out of bounds. pipeline.py's normalizer-fitting pass reads only cond_cont but had to be handed the maps to satisfy that. No parameter, buffer or state_dict change; existing checkpoints load unchanged, and the protected migration surfaces are untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 14:16:20 +02:00
lars	b63edcb8f9	Merge pull request 'Deduplicate n_sec_head/type_head MLPs into build_mlp_head (gitea #36)' (#53) from fix/issue-36 into master ... Reviewed-on: #53	2026-08-14 11:04:59 +02:00
lars	593c5f4d34	Deduplicate n_sec_head/type_head MLPs into build_mlp_head (gitea #36) ... The same two-layer classifier head (Linear(cond_out_dim, hidden_dim // 2) -> SiLU -> Linear(hidden_dim // 2, out_dim)) was hand-rolled five times in giant/model/models.py: Stage1Model.n_sec_head, Stage2OneShot.n_sec_head/ .type_head, and Stage2Autoregressive.n_sec_head/.type_head. The `// 2` ratio and fixed 2-layer depth were undocumented magic numbers, and both n_sec accuracy and secondary-species accuracy are known weak spots that were untunable independently of the trunk they hang off. Adds `build_mlp_head(in_dim, out_dim, hidden, depth, act)` to giant/model/layers.py (depth=1 is a bare Linear; depth>=2 matches the old hardcoded shape exactly), and a new `HeadConfig` (hidden_ratio, depth) dataclass in giant/config.py, wired in as `stage1_model.heads.n_sec` and `stage2_model.heads.{n_sec,type}` — split per head type (not one shared block per stage) since n_sec and species prediction are called out as separate weak spots that may want independent capacity. Defaults (hidden_ratio=0.5, depth=2) reproduce the old hardcoded architecture bit-for-bit, so every existing config.toml and migrated v0.2 checkpoint is unaffected; no changes were needed to migrate_config or the legacy migration surfaces. No new CLI flags, matching how other nested sub-config (router.*, trunk.*) is set via config.toml rather than per-field flags. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 10:57:47 +02:00
lars	c00ee91a74	Merge pull request 'Make HistoryEncoder a pluggable registry, like Router/Objective (gitea #35)' (#52) from fix/issue-35 into master ... Reviewed-on: #52	2026-08-14 10:43:05 +02:00
lars	f301fd98d2	Make HistoryEncoder a pluggable registry, like Router/Objective (gitea #35) ... Stage2Autoregressive.init_history_cache and .history_step both isinstance-checked self.history_encoder against AttentionHistory to decide whether to use its real incremental-cache methods or a no-op fallback, so a third history type couldn't be added without editing Stage2Autoregressive itself. The two-value "markov"/"attention" enum was also independently hardcoded in three places (Stage2Autoregressive's own validation, config.py's validate_config, and AutoregressiveConfig.from_dict's default). Mirrors the Router (giant/model/routers.py) and Objective (giant/model/objectives.py, gitea #32) pattern: HistoryEncoder now declares working O(1) init_cache/step defaults (init_cache -> None, step -> one forward() call), so every registered history type satisfies the incremental interface without opting in; AttentionHistory overrides both with its real KV-cache versions since its forward() needs the full prefix. Added HISTORY_REGISTRY/register_history/build_history, registered "markov" and "attention", and deleted both isinstance checks in models.py. Per user decision during planning, config.py's validate_config now imports HISTORY_REGISTRY and checks membership dynamically instead of keeping its own hardcoded tuple, making the registry the single source of truth end to end (verified no import cycle: config.py had no prior dependency on giant.model, and giant.model.history has none on giant.config). No config-schema change and no checkpoint impact — this is a pure internal-interface refactor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 10:35:46 +02:00
lars	9752ddf79c	Merge pull request 'Add an Objective registry for the flow/ddpm/wgan generator choice (gitea #32)' (#51) from fix/issue-32 into master ... Reviewed-on: #51	2026-08-14 10:22:51 +02:00
lars	f8722e347e	Add an Objective registry for the flow/ddpm/wgan generator choice (gitea #32) ... generator ∈ {"flow", "ddpm", "wgan"} was tested as a bare string in ~45 sites across models.py, sample.py, builders.py, trainers.py, and stage2_inputs.py, each independently re-deriving one of five consequences of the choice (needs a time embedding? what does the trunk take as input? is the type slice folded into the trunk output? which sampler? which loss?). giant/model/objectives.py adds an Objective ABC + OBJECTIVE_REGISTRY + build_objective factory, mirroring routers.py's Router pattern, and every bare-string site now goes through it (needs_time, is_adversarial, folds_type_slice, trunk_in_dim, build_schedule, stage1_loss/stage2_loss). Per discussion: FlowDDPMStageTrainer and WGANStageTrainer stay separate classes rather than merging into one StageTrainer as the issue's sketch proposed — their training loops are genuinely different shapes (single loss vs. dual G/D step with gradient penalty/n_critic/ST-Gumbel), and trainers.py is the least-covered-by-fast-tests part of the codebase, so a full merge was judged out of proportion to this issue's risk budget. FlowDDPMStageTrainer's own loss dispatch (flow vs ddpm, one-shot vs AR) does move onto the objective, so a future non-adversarial objective (rectified flow, consistency distillation) is still a one-file, zero-trainer-edits addition. No config-schema change — stage{1,2}_model.generator stays the persisted string, just looked up in the registry instead of string-compared. An unrecognized generator value now fails fast with a clear ValueError instead of silently falling through some bare-string checks and not others (same behavior build_router/build_trunk already have for their own type keys). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 10:10:58 +02:00
lars	c8f52259d6	Merge pull request 'Make ResBlock's conditioning-injection mechanism selectable (gitea #34)' (#49) from fix/issue-34 into master ... Reviewed-on: #49	2026-08-14 09:52:17 +02:00
lars	0f95e0eaae	Make ResBlock's conditioning-injection mechanism selectable (gitea #34) ... ResBlock injected conditioning exactly one way — h = linear1(h) + cond_proj(cond), a conditional bias, the weakest standard option for a model whose entire job is to be conditional. Adds BLOCK_REGISTRY (giant/model/layers.py), mirroring the TRUNK_REGISTRY/ROUTER_REGISTRY registry+factory idiom (gitea #33), with two new drop-in alternatives: FilmResBlock (per-channel scale+shift modulating the norm output, zero-init so conditioning has no effect at construction) and AdaLNResBlock (DiT-style AdaLN-Zero — the norm's own affine is replaced by a conditioning-derived scale/shift, plus a zero-init gate on the residual branch, making the block the exact identity function at init). Selected per stage via a new stage{1,2}_model.trunk.block_conditioning config leaf ("add" | "film" | "adaln", default "add"), threaded through build_trunk/build_expert_body/RoutedTrunk and the three stage model constructors. Default stays "add" and ResBlock's body is unchanged, so existing configs/checkpoints are bit-identical to before this change. Decided during planning: the new field lives on the existing TrunkConfig rather than a new top-level block/blocks config section; the WGAN CriticModel (which builds its own ResBlock stack outside TRUNK_REGISTRY) and the issue's mentioned blocks.norm/blocks.activation axes are both left out of scope. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 09:41:50 +02:00
lars	dc4cad7d11	Merge pull request 'Make trunk architecture selectable via a registry (gitea #33)' (#48) from fix/issue-33 into master ... Reviewed-on: #48	2026-08-14 09:24:32 +02:00
lars	f3f7645bf7	Make trunk architecture selectable via a registry (gitea #33) ... build_trunk hardcoded exactly two shapes (MonolithicTrunk/RoutedTrunk), chosen only by whether a Router was built, with no way to select a different trunk body architecture at all. Deviates from the issue's literal proposal (a TRUNK_REGISTRY choosing between "resmlp"/"moe" trunk shapes): during planning, decided that the trunk *body* architecture and whether it's *mixed* are orthogonal, so the registry (TRUNK_REGISTRY/register_trunk/build_expert_body in giant/model/trunks.py) holds expert bodies only (today: "resmlp", ExpertTrunk's existing input_proj -> ResBlock stack -> out_proj). Routing stays exactly router.enabled/n_experts, untouched — a future transformer body gets a mixture variant for free (trunk.type = "transformer" + router.enabled = true) instead of needing a separate registry entry per (body x routed/not) combination. MonolithicTrunk is deleted; the unrouted case now returns the registry-selected body directly, preserving today's exact state-dict keys (trunk.input_proj.* etc., not trunk.experts.0.*) — required both for existing non-routed checkpoints and because _legacy.py's migrate_legacy_state_dict already assumes that flat layout for a v0.2 checkpoint. New config leaf only: stage{1,2}_model.trunk.type: str = "resmlp" (TrunkConfig). hidden_dim/n_res_blocks/dropout stay where they are today. Nothing about router.enabled, config.migrate_config, _legacy.py, or the CLI's --router flags changes — a v0.2-migrated config gets trunk.type = "resmlp" automatically, reproducing current behaviour exactly. No CLI flag added (matches the config.toml-only precedent set by autoregressive.history/particle_type.target/n_sec.mode). No transformer body and no "none"/"linear" body (gitea #45) in this change. Full design rationale recorded on gitea #33 and #45 before implementation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-14 09:16:55 +02:00
lars	c83e72b689	Merge pull request 'V0.3.0 stage2 autoregressive' (#27) from v0.3.0-stage2-autoregressive into master ... Reviewed-on: #27 v0.3.0	2026-08-13 16:27:32 +02:00
lars	f505fe7f22	Skip router auxiliary loss compute when their lambda is 0 (gitea #31) ... FlowDDPMStageTrainer._compute unconditionally called router.balance_loss/classify_loss/entropy_loss whenever a router existed, then only added each term into total if its lambda was > 0 -- so every routed run paid for balance_loss/entropy_loss's extra router.gate(...) forward passes even at the default lambda_balance = lambda_proc = lambda_entropy = 0.0 (the exact config the failed 2026-07-22 router benchmark ran). Guard each computation on the same > 0 condition that already guarded the addition, matching WGANStageTrainer's cost structure which has no router-loss block at all. total's value is unchanged either way. Added a test that spies on the router's three loss methods and checks call counts both at lambda=0 (must be skipped) and lambda>0 (must still run, so the guard doesn't suppress the real path). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-13 16:18:19 +02:00
lars	32aa5a5f92	Decouple secondary-species vocabulary from conditioning.particle.emb_dim (gitea #29) ... conditioning.particle.emb_dim and stage2_model.particle_type.target="onehot"'s class count were silently the same number everywhere (pipeline.py's PDG top-N map build, Stage2OneShot/Stage2Autoregressive's type head, StageSpec's training loss width, the checkpoint's shared pdg_topn_map), fixing the secondary-species vocabulary at whatever width the unrelated physical-conditioning MLP happened to use — the exact vocabulary the v0.3.0 pivot exists to fix. Adds stage2_model.particle_type.n_classes (default 0 = inherit conditioning.particle.emb_dim, preserving today's behavior and every existing checkpoint) and a single resolve_type_n_classes helper used everywhere the coupling used to be implicit. Splits the checkpoint's shared pdg_topn_map into a conditioning-only pdg_topn_map and a new sec_type_topn_map, built independently through the existing (axis, n_classes)-keyed setup cache (no extra scan when they still resolve to the same N) and threaded through giant predict/giant rollout's decode path. A checkpoint with no sec_type_topn_map key (pre-#29) falls back to reusing pdg_topn_map, reproducing the old shared behavior exactly. Decided with the user during planning: commit directly on this branch; represent the split as an additive sec_type_topn_map checkpoint key rather than conditionally reusing pdg_topn_map; build the two top-N maps independently rather than the issue's proposed build-at-max-and-slice, since the setup cache already avoids redundant scans across runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-13 16:11:14 +02:00
lars	899ca3a7d5	Validate stage2_model.autoregressive.order in validate_config (gitea #30) ... order was documented as single-valued ("energy_desc" only, placeholder for a future alternative ordering) but validate_config only checked its siblings history/teacher_forcing, so e.g. order = "energy_asc" was silently accepted and trained as if it were energy_desc. Add the missing check alongside the other two, gated the same way (only meaningful under stage2_model.decoder = "autoregressive"). Also updates the stale reason string on the pre-existing _KNOWN_UNUSED allow-list entry for this key in tests/test_config_consumed_keys.py, since half of it ("validate_config ... never [checks] order") is no longer true after this fix — the key stays allow-listed because validate_config itself isn't in that test's build/train/rollout consumer whitelist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-13 15:40:47 +02:00
lars	da717971b6	Honour wgan.critic_hidden_dim/critic_n_res_blocks in build_critics (gitea #28) ... build_critics always sized a WGAN critic off the generator's own hidden_dim/n_res_blocks, silently discarding the documented 0=inherit sentinel on stage{1,2}_model.wgan.critic_hidden_dim/critic_n_res_blocks (the same convention critic_lr already honoured). Now both keys are read with the 0 -> inherit fallback, and stage-scoped-only CLI flags (--stage{1,2}-critic-hidden-dim/--stage{1,2}-critic-n-res-blocks) are added -- no shared alias, since critic sizing is an architectural per-stage knob like --hidden-dim/--n-res-blocks, not a shared training hyperparameter like --n-critic/--gp-weight/--noise-dim/--critic-lr. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-13 15:36:26 +02:00
lars	c3fc768b40	Reject stage2_model.stage1_context = 'sampled' as unimplemented (issues.md Issue 1) ... trainers.py unconditionally trains stage 2 against the ground-truth stage-1 output (stage1_ctx = x1_s1.detach()), but 'sampled' was accepted by validate_config, stored in config.toml and the checkpoint's model_config, and silently trained identically to 'truth' — mislabeling every downstream artifact for a run launched with --stage2-stage1-context sampled. Mirrors the existing stop_token validate_config pattern. User chose the immediate fix (reject loudly) over the proper fix (actually implement sampled context), which is scoped to Issue 16. Also updates the _KNOWN_UNUSED reason for stage2_model.stage1_context (added by Issue 5's consumed-keys audit) to reflect that the value is now rejected rather than silently accepted, and drops the now-invalid --stage2-stage1-context sampled case from test_stage2_only_knobs (a full CLI invocation) — that flag's plumbing is still covered at the overrides-dict level by test_overrides_from_flags_stage2_only_knobs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-13 14:57:21 +02:00
lars	a4b5a6c3bf	Add consumed-keys audit test (issues.md Issue 5) ... validate_config_keys only checks that a config key is declared in DEFAULT_CONFIG, never that anything reads it — the gap that let Issues 1, 2 and 4's dead keys (stage1_context, wgan.critic_hidden_dim/critic_n_res_blocks, autoregressive.order) slip through silently. tests/test_config_consumed_keys.py walks every DEFAULT_CONFIG leaf path and asserts each is either found (via AST scan for attribute access, dict-key-shaped string constants, or constructor/ function parameter names — the last needed because Router subclasses receive their config via **kwargs filtered by signature) in a fixed whitelist of build/train/rollout consumer files, or explicitly recorded in _KNOWN_UNUSED with a reason. A second test asserts the allow-list has no stale entries, so fixing Issue 1/2/4 will force removal of the corresponding allow-list line rather than let it silently outlive the bug. The whitelist is intentionally narrower than "anywhere in giant/": scanning the whole package produces false negatives from unrelated identifier collisions (e.g. router_gating.py's unrelated `order` parameter would make autoregressive.order read as consumed). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-13 14:42:47 +02:00
lars	30a448927c	Remove issues.md ... All tracked issues have been resolved and merged individually. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-13 10:42:01 +02:00
lars	81eb14d75c	Move scripts/ to giant/tools/ (issues.md Issue 9) ... `scripts` was published as a top-level distribution package, colliding with one of the most generic names in the Python ecosystem and shadowable by a stray scripts/ dir on the portal machines' shared /work/lbogner. Move it under the giant namespace; the dwarf command name is unchanged, only the Python import path and file location move. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-13 10:31:47 +02:00
lars	72f5a891bf	Split giant/model/network.py into giant/model/ (issues.md Issue 8) ... Pure file-move refactor: network.py's 1742 lines held six distinct concerns (layers, condition encoder, routers, trunks, history encoders, stage models, legacy migration, builders) that the v0.3.0 composable-parts refactor already separated at the class level but not the file level. Split along those seams into layers.py/encoders.py/routers.py/trunks.py/ history.py/models.py/_legacy.py/builders.py; network.py is now an 83-line re-export shim so no external import site needed to change. No logic, signature, or behavior changes.	2026-08-13 10:21:13 +02:00
lars	a4f4cba58b	Type the data/model/training batch contracts with NamedTuples (issues.md Issue 7) ... build_features (transforms.py) now returns StepFeatures and StreamingStepsDataset (dataset.py) now yields StepBatch, both NamedTuples with the same field order as the tuples they replace, so ty can catch a dropped/added field at every consuming call site instead of a silent positional-tuple mismatch. Converted the unreadable throwaway-heavy unpacks in cli.py, pipeline.py, validate.py, and dataset.py to named attribute access; gave the WGAN path's derived 5-element batch its own _Stage2RealFakeBatch NamedTuple; updated the two test batch-construction helpers to build real StepBatchs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-13 10:11:58 +02:00
lars	e6261cea03	Unify the two v0.2->v0.3 migration surfaces (issues.md Issue 6) ... giant/config.py:migrate_config (config.toml) and giant/model/network.py:_migrate_legacy_model_config (checkpoint model_config) independently hand-maintained the same v0.2 facts and an identical router expert-sizing rejection. Extract the shared knowledge into a new leaf module, giant/_migration.py (V02_MODEL_KEY_TO_STAGES, V02_FIXED_FACTS, reject_legacy_router_expert_sizing), consumed by both. Also replace NSecConfig's legacy-only, nullable legacy_owner sentinel (living in an extra: dict catch-all) with a normal, always-set owner: str = "stage2" field, so build_models reads one concrete two-valued key instead of branching on a legacy marker. Record in CLAUDE.md that v0.2 checkpoint-loading support has no expiry decided yet, since /ceph still holds pre-v0.3.0 checkpoints. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-13 09:56:22 +02:00
lars	733c13c31c	Mark issues.md Issue 5 as fixed ... Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-12 15:31:17 +02:00