Add data-integrity guards against silent NaN/Inf propagation and races
CI / Lint (ruff check) (push) Successful in 33s
CI / Format (ruff format) (push) Successful in 34s
CI / Sync project version with tag (push) Has been skipped
CI / Type check (ty) (push) Successful in 32s
CI / Tests (push) Successful in 2m12s

- log_transform / _validate_unit_pre_dir now raise on non-finite input
  instead of letting a NaN row silently poison the persisted normalizer
  cache (norm < 1e-6 was always False for NaN, so the existing guard
  never caught it).
- encode_secondaries warns when a row's secondary energies cumulatively
  exceed e_sec, instead of silently saturating the overflowing slot's
  stick-breaking logit via the _EPS floor.
- EVENT_ID_FILE_STRIDE overflow now raises instead of silently colliding
  two files' event ids together (reintroducing train/val leakage).
- make_event_split(val_fraction=0.0) now actually holds out nothing,
  instead of always forcing at least 1 validation event.
- setup_cache.save() is now serialized with a flock, since two
  concurrent writers (a real scenario on this repo's shared
  portal/condor machines) could otherwise race and silently drop one
  writer's freshly-computed cache section.
- Documented (no behavior change) the pre_dir ≈ -ẑ antipodal rotation
  singularity in _rodrigues_axis, which is real but inherent to any
  single-valued local-frame convention.

Each fix has a regression test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-03 13:47:29 +02:00
parent a4c0443e01
commit 74343d3e48
8 changed files with 247 additions and 15 deletions
+35
View File
@@ -214,6 +214,41 @@ def test_save_merges_non_colliding_normalizer_keys(tmp_path):
assert loaded.normalizers["k2"].n_train_steps == 2
def test_save_is_serialized_against_concurrent_writers(tmp_path):
"""Without the flock in setup_cache.save(), two concurrent writers can
both load() the same base state and merge their own section in
independently, so whichever os.replace() lands last silently drops the
other's key — a lost-update race, not a corrupt file. Each of these
threads writes a distinct normalizer key many times over; if the
load-merge-write critical section isn't actually serialized, at least
one thread's key is likely to go missing from the final merged cache."""
import threading
data = _touch_parquet(tmp_path / "shard.parquet")
files = [data]
setup_cache.save(data, files, SetupCache.empty(files))
n_writers, n_rounds = 6, 15
def _writer(idx: int) -> None:
for r in range(n_rounds):
cache = SetupCache.empty(files)
cache.normalizers[f"k{idx}"] = _entry(n_train_steps=r)
setup_cache.save(data, files, cache)
threads = [threading.Thread(target=_writer, args=(i,)) for i in range(n_writers)]
for t in threads:
t.start()
for t in threads:
t.join()
loaded = setup_cache.load(data, files)
assert loaded is not None
assert set(loaded.normalizers.keys()) == {f"k{i}" for i in range(n_writers)}
for i in range(n_writers):
assert loaded.normalizers[f"k{i}"].n_train_steps == n_rounds - 1
# ── energy_quantiles_from_sample / energy_quantile_at ───────────────────