fix(analysis): keep pre_E alive through prediction range subsampling
CI / Sync project version with tag (hand-pushed tags only) (pull_request) Skipped
CI / Publish package to Gitea package registry (pull_request) Skipped
CI / Format (ruff format) (pull_request) Successful in 2m23s
CI / Lint (ruff check) (pull_request) Successful in 2m33s
CI / Type check (ty) (pull_request) Successful in 2m51s
CI / Tests (pull_request) Successful in 6m35s
CI / Release (bump, changelog, badges, tag) on merge to master (pull_request) Skipped

`_row_subsample` hash-filters on `pre_E`, but `build_context`'s
prediction branch called it on a frame already projected down to
`pred_<var>`/`true_<var>` columns, so any run whose prediction file
exceeds `sample_rows` (the default is 1M; real predict outputs can be
100M+ rows) failed with `ColumnNotFoundError: pre_E`.

Subsample the full paired frame first, then project — matching every
other _row_subsample call site — and restructure the loop to subsample
once per prediction side instead of once per paired variable, cutting
6 streaming passes over the prediction file down to 1.

Add a regression test with sample_rows below the fixture's row count
so the hash-filter branch is actually exercised (the existing test
never took it).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qb7xBAa6aAR94AzimgpPxq
This commit is contained in:
2026-09-09 14:23:42 +02:00
co-authored by Claude Sonnet 5
parent 3864e5249b
commit c0cbc99231
2 changed files with 49 additions and 14 deletions
+22
View File
@@ -193,6 +193,28 @@ def test_build_context_resolves_prediction_ranges():
assert ctx.pred_top_sec_pdgs # secondaries present in the fixture
def test_build_context_resolves_prediction_ranges_when_subsampled():
"""Regression test: `sample_rows` smaller than the prediction row count
must still work — `_row_subsample`'s hash key (`pre_E`) has to survive
into the prediction branch's subsample call, not be projected away first
(see giant/analysis/context.py's predictions loop)."""
ctx = build_context(
[RolloutSpec("rollout", _rollout_frame())],
_reference_frame(),
predictions=[PredictionSpec("pred", _global_prediction_frame())],
n_energy_bins=2,
n_marginal_bins=10,
top_k_pdg=3,
sample_rows=2, # < the 3-row prediction fixture: forces the hash-filter branch
seed=0,
)
for var in PAIRED_SCALARS + ("cos_scatter", "cos_travel"):
lo, hi = ctx.pred_var_ranges[var]
assert np.isfinite(lo) and np.isfinite(hi) and lo < hi
lo, hi = ctx.pred_residual_ranges[var]
assert np.isfinite(lo) and np.isfinite(hi) and lo < hi
def test_prediction_specs_compute_valid_reduced():
ctx = _ctx_with_predictions()
bundle = Bundle.open(