lars/giant
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Let dwarf warm-cache take --config so it can't under-warm a config's cache keys (gitea #59) #60

Merged
lars merged 1 commits from fix/issue-59 into master 2026-08-17 09:24:44 +02:00
Conversation 0 Commits 1 Files Changed 3
+197 -66
lars commented 2026-08-17 09:16:04 +02:00
Owner
Copy Link
Copy Source

warm-cache built its config from DEFAULT_CONFIG with only a handful of
flags overridable, so it had no way to express settings like
stage2_model.particle_type.n_classes. configs/baseline.toml sets that
to 32; warm-cache always warmed the pdg top-N map under the emb_dim
default (16) instead, so a giant train --config configs/baseline.toml
run silently missed the cache and repaid the full parquet scan
warm-cache exists to avoid.

warm-cache now accepts the same --config a training run takes and
resolves every value run_setup_stage needs (val_fraction/seed,
conditioning types, both stages' router, particle_type.n_classes, ...)
from one gconfig.merge_cli_overrides + validate_config pass, exactly
like giant train's own pipeline does — so warming and training are
guaranteed to agree. Per user decision, --config is mutually exclusive
with the individual --val-fraction/--seed/--particle-conditioning/
--material-conditioning/--router*/flags (rejected outright rather than
silently layered on top), since a hardcoded CLI default clobbering an
unset config value is the same failure mode one level down. Also drops
a hardcoded stage2_model.router/k_max override that was a no-op against
today's defaults but would have clobbered a config setting either one
away from its default — same bug class.

Adding validate_config surfaced that the existing
test_warm_cache_router_process_warms_proc_map test was warming a
router.type="process" + conditioning.particle.type="physical" (the
CLI's old hardcoded default) combination that giant train's own
validate_config would already reject as incompatible — fixed by
passing --particle-conditioning embedding, which is what a working
--router-type process run actually requires.

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com

warm-cache built its config from DEFAULT_CONFIG with only a handful of flags overridable, so it had no way to express settings like stage2_model.particle_type.n_classes. configs/baseline.toml sets that to 32; warm-cache always warmed the pdg top-N map under the emb_dim default (16) instead, so a `giant train --config configs/baseline.toml` run silently missed the cache and repaid the full parquet scan warm-cache exists to avoid. warm-cache now accepts the same --config a training run takes and resolves every value run_setup_stage needs (val_fraction/seed, conditioning types, both stages' router, particle_type.n_classes, ...) from one gconfig.merge_cli_overrides + validate_config pass, exactly like giant train's own pipeline does — so warming and training are guaranteed to agree. Per user decision, --config is mutually exclusive with the individual --val-fraction/--seed/--particle-conditioning/ --material-conditioning/--router*/flags (rejected outright rather than silently layered on top), since a hardcoded CLI default clobbering an unset config value is the same failure mode one level down. Also drops a hardcoded stage2_model.router/k_max override that was a no-op against today's defaults but would have clobbered a config setting either one away from its default — same bug class. Adding validate_config surfaced that the existing test_warm_cache_router_process_warms_proc_map test was warming a router.type="process" + conditioning.particle.type="physical" (the CLI's old hardcoded default) combination that giant train's own validate_config would already reject as incompatible — fixed by passing --particle-conditioning embedding, which is what a working --router-type process run actually requires. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
lars added 1 commit 2026-08-17 09:16:06 +02:00
Let dwarf warm-cache take --config so it can't under-warm a config's cache keys (gitea #59)
CI / Format (ruff format) (push) Successful in 40s
Details
CI / Lint (ruff check) (push) Successful in 40s
Details
CI / Sync project version with tag (push) Has been skipped
Details
CI / Lint (ruff check) (pull_request) Successful in 42s
Details
CI / Type check (ty) (push) Successful in 44s
Details
CI / Format (ruff format) (pull_request) Successful in 36s
Details
CI / Sync project version with tag (pull_request) Has been skipped
Details
CI / Type check (ty) (pull_request) Successful in 37s
Details
CI / Tests (push) Successful in 3m48s
Details
CI / Tests (pull_request) Successful in 3m44s
Details
d25dfc0343 
warm-cache built its config from DEFAULT_CONFIG with only a handful of
flags overridable, so it had no way to express settings like
stage2_model.particle_type.n_classes. configs/baseline.toml sets that
to 32; warm-cache always warmed the pdg top-N map under the emb_dim
default (16) instead, so a `giant train --config configs/baseline.toml`
run silently missed the cache and repaid the full parquet scan
warm-cache exists to avoid.

warm-cache now accepts the same --config a training run takes and
resolves every value run_setup_stage needs (val_fraction/seed,
conditioning types, both stages' router, particle_type.n_classes, ...)
from one gconfig.merge_cli_overrides + validate_config pass, exactly
like giant train's own pipeline does — so warming and training are
guaranteed to agree. Per user decision, --config is mutually exclusive
with the individual --val-fraction/--seed/--particle-conditioning/
--material-conditioning/--router*/flags (rejected outright rather than
silently layered on top), since a hardcoded CLI default clobbering an
unset config value is the same failure mode one level down. Also drops
a hardcoded stage2_model.router/k_max override that was a no-op against
today's defaults but would have clobbered a config setting either one
away from its default — same bug class.

Adding validate_config surfaced that the existing
test_warm_cache_router_process_warms_proc_map test was warming a
router.type="process" + conditioning.particle.type="physical" (the
CLI's old hardcoded default) combination that giant train's own
validate_config would already reject as incompatible — fixed by
passing --particle-conditioning embedding, which is what a working
--router-type process run actually requires.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
lars merged commit ff435883ed into master 2026-08-17 09:24:44 +02:00
lars deleted branch fix/issue-59 2026-08-17 09:24:44 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
architecture
bug
Silent/wrong behavior, no crash
 experiment
feature
New feature matching the v0.3 redesign
 investigation
modularity
Making the network genuinely pluggable
 performance
phase2
validation
wontfix
Not being fixed right now
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
lars (Lars Bogner)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: lars/giant#60
Delete Branch "fix/issue-59"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?