Add sampled n_sec under n_sec.mode = 'head' (gitea #86) #88

Merged

lars merged 1 commits from fix/issue-86 into master 2026-08-28 11:18:07 +02:00

Conversation 0 Commits 1 Files Changed 7

+165 -37

lars commented 2026-08-28 11:08:05 +02:00

Owner

Copy Link

Copy Source

Taking argmax over the n_sec classifier logits collapses secondary
multiplicity onto its conditional mode at fixed pre-step conditioning,
under-dispersing n_sec in rollouts and biasing low wherever the true
conditional count distribution is right-skewed (typical for
multiplicity).

Generalizes stage2_model.n_sec.stop_sampling (previously stop_token-only)
into stage2_model.n_sec.sampling, covering both "head" (greedy: argmax;
sample: categorical draw via torch.multinomial) and "stop_token" (unchanged:
greedy threshold / Bernoulli draw) modes. stop_sampling is kept as a
deprecated alias in NSecConfig.from_dict and migrate_config, since it
appears in existing checkpoints' model_config. Default stays "greedy" so
existing runs/checkpoints are unaffected.

Taking argmax over the n_sec classifier logits collapses secondary multiplicity onto its conditional mode at fixed pre-step conditioning, under-dispersing n_sec in rollouts and biasing low wherever the true conditional count distribution is right-skewed (typical for multiplicity). Generalizes stage2_model.n_sec.stop_sampling (previously stop_token-only) into stage2_model.n_sec.sampling, covering both "head" (greedy: argmax; sample: categorical draw via torch.multinomial) and "stop_token" (unchanged: greedy threshold / Bernoulli draw) modes. stop_sampling is kept as a deprecated alias in NSecConfig.from_dict and migrate_config, since it appears in existing checkpoints' model_config. Default stays "greedy" so existing runs/checkpoints are unaffected.

lars added 1 commit 2026-08-28 11:08:06 +02:00

Add sampled n_sec under n_sec.mode = 'head' (gitea #86) ... fcd77c2f4b

Taking argmax over the n_sec classifier logits collapses secondary
multiplicity onto its conditional mode at fixed pre-step conditioning,
under-dispersing n_sec in rollouts and biasing low wherever the true
conditional count distribution is right-skewed (typical for
multiplicity).

Generalizes stage2_model.n_sec.stop_sampling (previously stop_token-only)
into stage2_model.n_sec.sampling, covering both "head" (greedy: argmax;
sample: categorical draw via torch.multinomial) and "stop_token" (unchanged:
greedy threshold / Bernoulli draw) modes. stop_sampling is kept as a
deprecated alias in NSecConfig.from_dict and migrate_config, since it
appears in existing checkpoints' model_config. Default stays "greedy" so
existing runs/checkpoints are unaffected.

lars merged commit 73975a4587 into master 2026-08-28 11:18:07 +02:00

lars deleted branch fix/issue-86 2026-08-28 11:18:09 +02:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

architecture

bug

Silent/wrong behavior, no crash

experiment

feature

New feature matching the v0.3 redesign

investigation

modularity

Making the network genuinely pluggable

performance

phase2

validation

wontfix

Not being fixed right now

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

lars (Lars Bogner)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: lars/giant#88