lars
5c576fa8f3
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 1m2s
CI / Lint (ruff check) (pull_request) Successful in 1m6s
CI / Format (ruff format) (pull_request) Successful in 1m6s
CI / Tests (pull_request) Successful in 2m46s
CI / Bump version, tag, and update changelog on merge to master (pull_request) Has been skipped
CI / Publish package to Gitea package registry (pull_request) Has been skipped
CI / Update README badges (version, test count) (pull_request) Has been skipped
sample_secondaries_ar ran all k_max=15 slots for every row regardless of each row's own predicted secondary count, even though the baseline checkpoint's rollout measured only 0.382 secondaries/step — so ~97% of stage-2 model calls generated tokens sec_valid then masked away. Compact the loop to the still-active row set at each slot: drop a row the moment its n_sec_pred is exhausted (or, under n_sec.mode="stop_token", the moment its own stop logit fires), so slot k's model calls cost O(active rows) instead of O(B). Exact — rows are independent given their own history — verified by comparing the compacted path against a new full_length=True escape hatch that reproduces the original uncompacted behavior bit-for-bit under deterministic noise. full_length=True is required by _assemble_stage2_ar_inputs_scheduled's scheduled-sampling self-sample, whose training contract needs a real prediction at every slot up to k_max regardless of a row's own count, so training behavior is unchanged. AttentionHistory's KV cache and MarkovHistory's O(1) state are kept aligned to the shrinking active set via a new HistoryEncoder.select_cache / Stage2Autoregressive.select_history_cache. Also fixes a latent bug the refactor surfaced: derived_n_sec (stop-token mode) could be overwritten by a later spurious re-fire of the stop logit on a row that had already stopped; now tracked via an explicit `finished` mask so only the first stop slot is recorded, matching the documented contract. No architecture or checkpoint-format change — every existing v0.3.0 Stage2Autoregressive checkpoint (flow/wgan, markov/attention, head/stop_token) picks up the speedup automatically on its next `giant rollout`/`giant predict`, no retraining needed. Measured (CPU, hidden_dim=512/6 blocks, k_max=15, batch 512, mean n_sec≈0.38 matching the baseline checkpoint's own rollout): 17.6-22.9x fewer wall-clock seconds for the AR loop alone (attention/markov history respectively). Directional only — baseline.toml's GPU inference-cost comment is updated accordingly, flagged stale pending a real rollout re-measurement via eval_cost_per_step. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HPt7bVLZYFJe5cG6V7ahqC