The categorical type path is not differentiable, and v0.3.0 accepts that:
the expected contribution of the broken path to the total gradient is
assumed negligible. Record it as an assumption with an explicit obligation
to demonstrate it, not as a settled result.
Separates the three things "broken" covers, since they have different
status: per-token loss under teacher forcing is fine (softmax CE needs no
sampling); ST-Gumbel into the critic is biased rather than absent (hard
forward, soft backward); full shower-rollout backprop was already
structurally non-differentiable once secondaries branch, so the switch
costs nothing that was not already lost. The accepted claim concerns only
the middle one.
Lists three ways to falsify it, cheapest first: gradient-magnitude
accounting through the type slice vs the continuous slices, a
detached-type ablation, and an estimator swap against REINFORCE if those
are inconclusive. The first is wired into implementation step 5 so
evidence accrues during the architecture comparison rather than in a
dedicated run afterwards, and the fallback if the ratio is not small is a
config change (target = "physical" or a non-adversarial CE head), not a
redesign.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lars
a489991a3b