Implement stage2_model.stage1_context = "sampled" #41

New Issue

2026-08-13T15:08:01+02:00

lars commented

2026-08-13 15:08:01 +02:00

Companion to the now-fixed silent no-op (stage2_model.stage1_context = "sampled" was previously accepted and silently trained identically to
"truth"; it's now rejected in validate_config as unimplemented). This issue
is the proper fix: actually implement "sampled".

Stage 2 is trained on ground-truth stage-1 outcomes and deployed on sampled
ones, and in a rollout that gap compounds over every step of every track in the
shower. This is the same train/inference gap that teacher_forcing = "scheduled" was added to close within stage 2 — the stage boundary just
never got the same treatment.

The plumbing exists: sample_stage1 is written, and when both stages are active
the training loop already holds both models. Suggest mirroring the scheduled
sampling precedent (ctx_p_start / ctx_p_end) rather than a hard switch, so
stage 2 doesn't chase a wildly moving target in early epochs.

Migrated from issues.md (v0.3.0 branch review, 2026-08-13), Issue 16.

Companion to the now-fixed silent no-op (`stage2_model.stage1_context = "sampled"` was previously accepted and silently trained identically to `"truth"`; it's now rejected in `validate_config` as unimplemented). This issue is the proper fix: actually implement `"sampled"`. Stage 2 is trained on ground-truth stage-1 outcomes and deployed on sampled ones, and in a rollout that gap compounds over every step of every track in the shower. This is the same train/inference gap that `teacher_forcing = "scheduled"` was added to close *within* stage 2 — the stage boundary just never got the same treatment. The plumbing exists: `sample_stage1` is written, and when both stages are active the training loop already holds both models. Suggest mirroring the scheduled sampling precedent (`ctx_p_start` / `ctx_p_end`) rather than a hard switch, so stage 2 doesn't chase a wildly moving target in early epochs. --- Migrated from `issues.md` (v0.3.0 branch review, 2026-08-13), Issue 16.

lars added the feature label 2026-08-13 15:08:01 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: lars/giant#41