Add "none" variants for router, history, and trunk (gitea #45) #73

Merged

lars merged 1 commits from fix/issue-45 into master 2026-08-24 09:32:33 +02:00

Conversation 0 Commits 1 Files Changed 6

+195 -3

lars commented 2026-08-24 09:23:20 +02:00

Owner

Copy Link

Copy Source

Turns "is this component earning its parameters?" into a one-line
config flip for each of the three pluggable network components:

• router.type = "none" (NoneRouter, giant/model/routers.py): still
  builds n_experts expert trunks via RoutedTrunk, but replaces the
  learned gate with a uniform 1/n_experts weight for every row — no
  centers/embeddings/classifier. Distinct from router.enabled=false
  (which drops routing/mixing entirely): this isolates whether the
  learned routing signal specifically is earning its parameters,
  holding expert count fixed.

• stage2_model.autoregressive.history = "none" (NoHistory,
  giant/model/history.py): ignores feat/has_prev entirely and always
  returns zeros, ablating whether the AR decoder's history
  conditioning earns its parameters. Already validated for free by
  gitea #35's generic HISTORY_REGISTRY membership check.

• trunk.type = "linear" (LinearTrunk, giant/model/trunks.py): a bare
  nn.Linear(in_dim + cond_dim, out_dim) body, no ResBlock stack. Per
  gitea #33's design, this composes for free with router.enabled=true
  ("mixture of trivial linear experts").

Both blocking issues (#33 trunk registry, #35 pluggable history
encoder) are closed, so this was unblocked.

Turns "is this component earning its parameters?" into a one-line config flip for each of the three pluggable network components: - router.type = "none" (NoneRouter, giant/model/routers.py): still builds n_experts expert trunks via RoutedTrunk, but replaces the learned gate with a uniform 1/n_experts weight for every row — no centers/embeddings/classifier. Distinct from router.enabled=false (which drops routing/mixing entirely): this isolates whether the *learned routing signal* specifically is earning its parameters, holding expert count fixed. - stage2_model.autoregressive.history = "none" (NoHistory, giant/model/history.py): ignores feat/has_prev entirely and always returns zeros, ablating whether the AR decoder's history conditioning earns its parameters. Already validated for free by gitea #35's generic HISTORY_REGISTRY membership check. - trunk.type = "linear" (LinearTrunk, giant/model/trunks.py): a bare nn.Linear(in_dim + cond_dim, out_dim) body, no ResBlock stack. Per gitea #33's design, this composes for free with router.enabled=true ("mixture of trivial linear experts"). Both blocking issues (#33 trunk registry, #35 pluggable history encoder) are closed, so this was unblocked.

lars added 1 commit 2026-08-24 09:23:21 +02:00

Add "none" variants for router, history, and trunk (gitea #45) ... 732d5f1cd2

Turns "is this component earning its parameters?" into a one-line
config flip for each of the three pluggable network components:

- router.type = "none" (NoneRouter, giant/model/routers.py): still
  builds n_experts expert trunks via RoutedTrunk, but replaces the
  learned gate with a uniform 1/n_experts weight for every row — no
  centers/embeddings/classifier. Distinct from router.enabled=false
  (which drops routing/mixing entirely): this isolates whether the
  *learned routing signal* specifically is earning its parameters,
  holding expert count fixed.

- stage2_model.autoregressive.history = "none" (NoHistory,
  giant/model/history.py): ignores feat/has_prev entirely and always
  returns zeros, ablating whether the AR decoder's history
  conditioning earns its parameters. Already validated for free by
  gitea #35's generic HISTORY_REGISTRY membership check.

- trunk.type = "linear" (LinearTrunk, giant/model/trunks.py): a bare
  nn.Linear(in_dim + cond_dim, out_dim) body, no ResBlock stack. Per
  gitea #33's design, this composes for free with router.enabled=true
  ("mixture of trivial linear experts").

Both blocking issues (#33 trunk registry, #35 pluggable history
encoder) are closed, so this was unblocked.

lars merged commit 12689cf5b6 into master 2026-08-24 09:32:33 +02:00

lars deleted branch fix/issue-45 2026-08-24 09:32:34 +02:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

architecture

bug

Silent/wrong behavior, no crash

experiment

feature

New feature matching the v0.3 redesign

investigation

modularity

Making the network genuinely pluggable

performance

phase2

validation

wontfix

Not being fixed right now

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

lars (Lars Bogner)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: lars/giant#73