fix(analysis): keep pre_E alive through prediction range subsampling
CI / Sync project version with tag (hand-pushed tags only) (pull_request) Skipped
CI / Publish package to Gitea package registry (pull_request) Skipped
CI / Format (ruff format) (pull_request) Successful in 2m23s
CI / Lint (ruff check) (pull_request) Successful in 2m33s
CI / Type check (ty) (pull_request) Successful in 2m51s
CI / Tests (pull_request) Successful in 6m35s
CI / Release (bump, changelog, badges, tag) on merge to master (pull_request) Skipped
CI / Sync project version with tag (hand-pushed tags only) (pull_request) Skipped
CI / Publish package to Gitea package registry (pull_request) Skipped
CI / Format (ruff format) (pull_request) Successful in 2m23s
CI / Lint (ruff check) (pull_request) Successful in 2m33s
CI / Type check (ty) (pull_request) Successful in 2m51s
CI / Tests (pull_request) Successful in 6m35s
CI / Release (bump, changelog, badges, tag) on merge to master (pull_request) Skipped
`_row_subsample` hash-filters on `pre_E`, but `build_context`'s prediction branch called it on a frame already projected down to `pred_<var>`/`true_<var>` columns, so any run whose prediction file exceeds `sample_rows` (the default is 1M; real predict outputs can be 100M+ rows) failed with `ColumnNotFoundError: pre_E`. Subsample the full paired frame first, then project — matching every other _row_subsample call site — and restructure the loop to subsample once per prediction side instead of once per paired variable, cutting 6 streaming passes over the prediction file down to 1. Add a regression test with sample_rows below the fixture's row count so the hash-filter branch is actually exercised (the existing test never took it). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qb7xBAa6aAR94AzimgpPxq
This commit is contained in:
@@ -193,6 +193,28 @@ def test_build_context_resolves_prediction_ranges():
|
||||
assert ctx.pred_top_sec_pdgs # secondaries present in the fixture
|
||||
|
||||
|
||||
def test_build_context_resolves_prediction_ranges_when_subsampled():
|
||||
"""Regression test: `sample_rows` smaller than the prediction row count
|
||||
must still work — `_row_subsample`'s hash key (`pre_E`) has to survive
|
||||
into the prediction branch's subsample call, not be projected away first
|
||||
(see giant/analysis/context.py's predictions loop)."""
|
||||
ctx = build_context(
|
||||
[RolloutSpec("rollout", _rollout_frame())],
|
||||
_reference_frame(),
|
||||
predictions=[PredictionSpec("pred", _global_prediction_frame())],
|
||||
n_energy_bins=2,
|
||||
n_marginal_bins=10,
|
||||
top_k_pdg=3,
|
||||
sample_rows=2, # < the 3-row prediction fixture: forces the hash-filter branch
|
||||
seed=0,
|
||||
)
|
||||
for var in PAIRED_SCALARS + ("cos_scatter", "cos_travel"):
|
||||
lo, hi = ctx.pred_var_ranges[var]
|
||||
assert np.isfinite(lo) and np.isfinite(hi) and lo < hi
|
||||
lo, hi = ctx.pred_residual_ranges[var]
|
||||
assert np.isfinite(lo) and np.isfinite(hi) and lo < hi
|
||||
|
||||
|
||||
def test_prediction_specs_compute_valid_reduced():
|
||||
ctx = _ctx_with_predictions()
|
||||
bundle = Bundle.open(
|
||||
|
||||
Reference in New Issue
Block a user