Store a quantile grid instead of a raw reservoir sample in the setup cache
CI / Format (ruff format) (push) Successful in 26s
CI / Lint (ruff check) (push) Successful in 28s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 24s
CI / Tests (push) Successful in 58s
CI / Format (ruff format) (pull_request) Successful in 28s
CI / Lint (ruff check) (pull_request) Successful in 28s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 28s
CI / Tests (pull_request) Successful in 59s
CI / Format (ruff format) (push) Successful in 26s
CI / Lint (ruff check) (push) Successful in 28s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 24s
CI / Tests (push) Successful in 58s
CI / Format (ruff format) (pull_request) Successful in 28s
CI / Lint (ruff check) (pull_request) Successful in 28s
CI / Sync project version with tag (pull_request) Has been skipped
CI / Type check (ty) (pull_request) Successful in 28s
CI / Tests (pull_request) Successful in 59s
NormalizerEntry.energy_reservoir_sample kept 100k raw energy values purely to seed EnergyRouter centers via np.quantile at load time, which alone accounted for most of the setup cache sidecar's ~2MB size (float32 values round-tripped through Python floats serialize at full double precision). Only a handful of quantile levels are ever read back, so collapse the sample to a fixed 1001-point quantile grid at save time and interpolate arbitrary levels from it at use time instead — about 100x smaller with negligible (<0.001) error on the levels that matter. Bumps the cache format version since old sidecars have no such grid to fall back on.
This commit is contained in:
@@ -30,7 +30,7 @@ def run_warm_setup_cache(
|
||||
`giant train` invocation will use so it hits this warmed entry.
|
||||
`router_enabled`/`router_type`/`n_experts` only matter for
|
||||
`router_type == "process"` (warms that `n_experts`'s process map); the
|
||||
energy-router reservoir sample is always collected regardless, so a
|
||||
energy-router quantile summary is always collected regardless, so a
|
||||
later `--router-type energy` run never needs to rescan just to seed
|
||||
centers.
|
||||
"""
|
||||
|
||||
Reference in New Issue
Block a user