Files
giant/tests
lars f3fec8bcb3 Add ComposedRouter for multi-axis MoE gating
Route on several independent axes at once (e.g. energy x pdg), each with
its own expert count and hyperparameters. The joint gate is the outer
product of per-axis softmax gates, so it stays a partition of unity and
top1/balance_loss factor per-axis. Config uses flat axis{i}_{field} keys
in model.router (TOML/CLI friendly), also settable via repeatable
--router-axis flags.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:53:21 +02:00
..