lars ad1b8e7835 Fix stale-partial reuse and n_chunks mismatch in analysis condor pipeline ...

- prep() now clears reduced_partial/ and reduced/ on every (re-)run.
  Partial files carry no record of what context (n_chunks, bin edges,
  group sets) they were computed under, so re-prepping the same run_dir
  with a different --chunks/--bins/--top-pdg (or after the rollout was
  regenerated) previously left old partials on disk that merge_one
  would silently merge against the new shared.json — producing a
  wrong-but-plausible reduced/*.json with no error.
- write_submit() now checks SubmitConfig.n_chunks against the run
  directory's own RunMeta.n_chunks (fixed at prep time, and what
  rows_per_chunk is sized against) and raises a clear error on
  mismatch, instead of an uncaught IndexError deep in _job_walltimes.

Each fix has a regression test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-08-03 13:48:43 +02:00

..

test_analysis_reduce.py

Rewrite analysis as streaming rollout-vs-reference plotting pipeline

2026-07-23 17:38:09 +02:00

test_bump_dataset_version.py

Fix ruff, ty, and pytest failures; apply ruff format

2026-07-02 16:59:22 +02:00

test_catalog.py

analyze: chunk per-plot aggregation across HTCondor jobs

2026-07-27 09:24:19 +02:00

test_cli_new_run.py

Add giant new-run to scaffold a config.toml + run dir ahead of training

2026-07-30 12:56:22 +02:00

test_cli_predict.py

Add --comment option to predict, recorded in YAML sidecar

2026-07-06 09:37:48 +02:00

test_condor.py

Fix stale-partial reuse and n_chunks mismatch in analysis condor pipeline

2026-08-03 13:48:43 +02:00

test_config.py

Add gumbel/learn_centers/learn_width/learn_temperature to out-dir naming

2026-07-30 16:42:56 +02:00

test_create_root_files.py

Support --energy-gev in dwarf make-root for the new minicalosim energy arg

2026-07-13 13:59:09 +02:00

test_dataset.py

Add data-integrity guards against silent NaN/Inf propagation and races

2026-08-03 13:47:29 +02:00

test_dwarf.py

Add dwarf warm-cache to precompute the setup-stage sidecar

2026-07-30 10:53:31 +02:00

test_flow.py

Implement Phase 2: secondary particle prediction

2026-06-29 11:34:31 +02:00

test_geometry.py

Add fast slab lookup for the GeometryOracle, replacing knn as the default

2026-07-08 11:42:12 +02:00

test_loader.py

Add data-integrity guards against silent NaN/Inf propagation and races

2026-08-03 13:47:29 +02:00

test_materials.py

Condition on material/particle physical properties instead of learned embeddings

2026-07-17 15:12:54 +02:00

test_network.py

Implement Phase 2: secondary particle prediction

2026-06-29 11:34:31 +02:00

test_particles.py

Fix crashes in physical-property conditioning edge cases

2026-07-20 10:36:25 +02:00

test_phase2.py

Speed up giant train's setup stage

2026-07-29 13:34:32 +02:00

test_pipeline.py

Fix training-loop checkpoint/resume and WGAN bugs

2026-08-03 13:48:22 +02:00

test_render.py

Rewrite analysis as streaming rollout-vs-reference plotting pipeline

2026-07-23 17:38:09 +02:00

test_rollout.py

Fix conditioning="physical" so it can actually generalize past training vocab

2026-08-03 13:47:59 +02:00

test_router_gating.py

analyze: add MoE router gating/share diagnostic plots

2026-07-24 13:32:25 +02:00

test_router.py

Fix conditioning="physical" so it can actually generalize past training vocab

2026-08-03 13:47:59 +02:00

test_setup_cache.py

Add data-integrity guards against silent NaN/Inf propagation and races

2026-08-03 13:47:29 +02:00

test_steps_to_parquet_parallel.py

Fix ruff, ty, and pytest failures; apply ruff format

2026-07-02 16:59:22 +02:00

test_steps_to_parquet.py

Apply ruff format after merging phase2-secondary-prediction

2026-07-15 10:00:36 +02:00

test_train.py

Scope wandb run config to only-active hyperparameters

2026-07-30 16:46:19 +02:00

test_transforms.py

Fix conditioning="physical" so it can actually generalize past training vocab

2026-08-03 13:47:59 +02:00

test_validate.py

Fix crashes in physical-property conditioning edge cases

2026-07-20 10:36:25 +02:00

test_wgan.py

Add WGAN-GP mode as a throwaway fast-eval experiment

2026-07-22 11:20:28 +02:00