Let dwarf warm-cache take --config so it can't under-warm a config's cache keys (gitea #59)
CI / Format (ruff format) (push) Successful in 40s
CI / Lint (ruff check) (push) Successful in 40s
CI / Sync project version with tag (push) Has been skipped
CI / Lint (ruff check) (pull_request) Successful in 42s
CI / Type check (ty) (push) Successful in 44s
CI / Format (ruff format) (pull_request) Successful in 36s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 37s
CI / Tests (push) Successful in 3m48s
CI / Tests (pull_request) Successful in 3m44s

warm-cache built its config from DEFAULT_CONFIG with only a handful of
flags overridable, so it had no way to express settings like
stage2_model.particle_type.n_classes. configs/baseline.toml sets that
to 32; warm-cache always warmed the pdg top-N map under the emb_dim
default (16) instead, so a `giant train --config configs/baseline.toml`
run silently missed the cache and repaid the full parquet scan
warm-cache exists to avoid.

warm-cache now accepts the same --config a training run takes and
resolves every value run_setup_stage needs (val_fraction/seed,
conditioning types, both stages' router, particle_type.n_classes, ...)
from one gconfig.merge_cli_overrides + validate_config pass, exactly
like giant train's own pipeline does — so warming and training are
guaranteed to agree. Per user decision, --config is mutually exclusive
with the individual --val-fraction/--seed/--particle-conditioning/
--material-conditioning/--router*/flags (rejected outright rather than
silently layered on top), since a hardcoded CLI default clobbering an
unset config value is the same failure mode one level down. Also drops
a hardcoded stage2_model.router/k_max override that was a no-op against
today's defaults but would have clobbered a config setting either one
away from its default — same bug class.

Adding validate_config surfaced that the existing
test_warm_cache_router_process_warms_proc_map test was warming a
router.type="process" + conditioning.particle.type="physical" (the
CLI's old hardcoded default) combination that giant train's own
validate_config would already reject as incompatible — fixed by
passing --particle-conditioning embedding, which is what a working
--router-type process run actually requires.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-17 08:54:14 +02:00
parent a1ecf0df1d
commit d25dfc0343
3 changed files with 197 additions and 66 deletions
+65
View File
@@ -124,6 +124,11 @@ def test_warm_cache_router_process_warms_proc_map(tmp_path):
[
"warm-cache",
str(data),
# router.type="process" is incompatible with the default
# conditioning.particle.type="physical" (validate_config, now
# enforced by warm-cache too — see gitea #59).
"--particle-conditioning",
"embedding",
"--router",
"--router-type",
"process",
@@ -167,3 +172,63 @@ def test_warm_cache_different_val_fraction_is_separate_entry(tmp_path):
assert loaded is not None
assert "valfrac=0.1_seed=0_pcond=physical_mcond=physical" in loaded.normalizers
assert "valfrac=0.3_seed=0_pcond=physical_mcond=physical" in loaded.normalizers
def test_warm_cache_config_warms_particle_type_n_classes(tmp_path):
"""gitea #59: a config setting stage2_model.particle_type.n_classes away
from its 0 (= inherit conditioning.particle.emb_dim) default must warm
the pdg top-N map under that n_classes, not the emb_dim default, so a
later `giant train --config <same file>` run hits it instead of quietly
re-scanning every parquet file."""
data = _make_synthetic_steps(tmp_path / "data.parquet", n_events=20)
config_path = tmp_path / "config.toml"
config_path.write_text("[meta]\nconfig_version = 3\n\n[stage2_model.particle_type]\nn_classes = 32\n")
runner.invoke(app, ["warm-cache", str(data), "--config", str(config_path)])
result = runner.invoke(app, ["warm-cache", str(data), "--config", str(config_path)])
assert result.exit_code == 0, result.output
assert "pdg top-N map: cache hit" in result.output
assert "32 classes" in result.output
def test_warm_cache_config_rejects_val_fraction_flag(tmp_path):
data = _make_synthetic_steps(tmp_path / "data.parquet", n_events=20)
config_path = tmp_path / "config.toml"
config_path.write_text("[meta]\nconfig_version = 3\n")
result = runner.invoke(
app,
["warm-cache", str(data), "--config", str(config_path), "--val-fraction", "0.2"],
)
assert result.exit_code != 0
assert "--config" in result.output
assert "--val-fraction" in result.output
def test_warm_cache_config_rejects_router_flags(tmp_path):
data = _make_synthetic_steps(tmp_path / "data.parquet", n_events=20)
config_path = tmp_path / "config.toml"
config_path.write_text("[meta]\nconfig_version = 3\n")
result = runner.invoke(
app,
[
"warm-cache",
str(data),
"--config",
str(config_path),
"--router",
"--router-type",
"process",
"--n-experts",
"3",
],
)
assert result.exit_code != 0
assert "--config" in result.output
assert "--router/--no-router" in result.output
assert "--router-type" in result.output
assert "--n-experts" in result.output