The dev extra already declares giant[workflow] in pyproject.toml but
uv.lock hadn't been regenerated to match. Also drop the uv cache prune
step from every CI job.
Replaces the extrapolated pre-v0.3 weak-spot claims in baseline.toml's
header (which had the wrong sign on step-count error) with measured
numbers from the first full rollout validation of this exact config,
and adds a matching Roadmap entry in CLAUDE.md.
torch/pandas/pyarrow/polars/uproot/awkward/particle were all imported
at module scope in giant/cli.py and giant/tools/dwarf.py, so even
`--help` paid ~1.6-1.9s of import cost. Move those imports into the
command bodies that actually need them (following the deferred-import
pattern already used for analysis/render/plots/sklearn/wandb), cutting
`giant --help` to ~0.3s and `dwarf --help` to ~0.2s with no change to
any command's actual behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A bare "pull_request:" key parses to null in YAML, which the runner
apparently doesn't treat as "trigger with defaults" the way an empty
mapping does — the open PR for this branch got no CI run at all.
- Drop [skip ci] from the bump-version/changelog/tag-sync commits so
master's tip always has a check run instead of only the merge commit.
- Filter those chore commits out of the changelog via message pattern
instead of the now-removed [skip ci] tag.
- Only run CI on push to master (plus tags); pull requests to any branch
still run the full suite.
- Add a publish-package job that builds and publishes to the Gitea PyPI
registry on every tag push, after tests and version sync pass.
giant predict/rollout rebuilt models straight from ckpt["model_config"] with
no way to change sampling-only keys (e.g. stage2_model.n_sec.stop_sampling)
without retraining. Adds config_overrides to load_for_inference, validated
against giant.config.INFERENCE_OVERRIDES so a typo or shape-bearing key
raises CheckpointCompatibilityError up front instead of an opaque
load_state_dict mismatch. Wired as a repeatable --set dotted.path=value on
both CLI commands, recorded in the rollout YAML sidecar, and surfaced in
`giant model summary`'s output.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Taking argmax over the n_sec classifier logits collapses secondary
multiplicity onto its conditional mode at fixed pre-step conditioning,
under-dispersing n_sec in rollouts and biasing low wherever the true
conditional count distribution is right-skewed (typical for
multiplicity).
Generalizes stage2_model.n_sec.stop_sampling (previously stop_token-only)
into stage2_model.n_sec.sampling, covering both "head" (greedy: argmax;
sample: categorical draw via torch.multinomial) and "stop_token" (unchanged:
greedy threshold / Bernoulli draw) modes. stop_sampling is kept as a
deprecated alias in NSecConfig.from_dict and migrate_config, since it
appears in existing checkpoints' model_config. Default stays "greedy" so
existing runs/checkpoints are unaffected.
Replace the event-level n_sec confusion matrix with two step-resolved
secondary-multiplicity comparisons:
- sec_count_per_step: overlay histogram of how many secondaries a single
step emits, rollout series vs reference.
- sec_count_per_step_by_species: heatmap of per-step multiplicity of one
species (zero row included) against species, drawn as one panel per
rollout plus a reference panel, raw counts on a log color scale.
Both are backed by a new sources.secondaries_by_step view, which tags each
secondary with its emitting step — (event_id, parent_id, birth position)
on the rollout side, the row index on the reference side — so neither plot
needs a join against the step frame. Steps that emitted nothing are
recovered by subtraction from the chunk's step count, keeping both specs
sum-mergeable across condor chunks.
The rollout multiplicity is derived from the actual secondary birth rows
rather than the n_sec_pred column, which records the predicted count
before the per-event max-tracks cap.
_render_heatmap gained reference-panel and log-color support;
marginal_distance_summary sets neither key and is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The uv cache lives on a persistent volume shared by every job on the
runner (/srv/act-runner-cache/uv), so nothing trimmed it and it grew
without bound. `uv cache prune --ci` drops the entries that are not worth
keeping between runs (pre-built wheels for local sources) while leaving
the downloaded-wheel cache that makes `uv sync` fast.
In bump-version the step carries the same is_merge guard as the rest of
that job, since uv is only set up on a merge push.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The per-epoch training fan-out only makes sense if epoch k is the same
epoch either way, and the shuffle fix alone wasn't enough: run_train_job
calls seed_everything(train.seed) at process start, so a fresh job
restarted the torch/numpy stream at epoch 1's state and drew different
flow/WGAN noise than the corresponding epoch of a single long run.
giant.config.epoch_seed derives a per-epoch seed, and the training loop
reseeds from it at the top of every epoch. Verified on a 3-epoch toy run:
the chained workflow's concatenated metrics.csv is now byte-identical to a
single `giant train --epochs 3` with the same seed (it matched only on
epoch 1 before).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
b2luigi's AnalysisComputeTask now submits the per-(plot, chunk) jobs, so the
bespoke submit-file generator has nothing left to do:
- giant/analysis/condor.py -> giant/analysis/run.py, dropping SubmitConfig,
the wrapper/submit-description templates, _job_walltimes and
_resolve_giant_executable. What stays is the actual logic — prep,
RunMeta, the rollout-YAML loading, compute_reduced/compute_one and
merge_one/merge_all — and the module no longer submits anything, hence
the name.
- `giant analyze submit` is gone; prep / compute-one / merge-one / list /
render / metrics remain as the single-step primitives the workflow calls.
- tests/test_condor.py -> tests/test_analysis_run.py, minus the
submit-description cases.
CLAUDE.md and README.md document the workflow package, the new `workflow`
extra, and — for whenever condor-gpu-train-rollout is merged — that its
train-submit/rollout-submit commands are deliberately superseded and must
not be revived.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One workflow TOML now parameterises a whole experiment and `giant workflow
run <spec.toml>` turns it into a b2luigi DAG whose targets are files on
/ceph: nothing already produced is recomputed, every step waits for its
inputs, and HTCondor submission/polling is b2luigi's job.
- spec.py: workflow TOML -> frozen dataclasses with name-uniqueness and
cross-reference validation, unknown keys rejected the way giant.config
rejects them, and a short spec_hash per task that folds in its transitive
parents — so an edited spec re-runs exactly the affected subtree.
- htcondor.py: the CPU/GPU submit settings. The GPU requirement strings
(ProvidesEtpCeph + optional device/memory pins) are ported from the
condor-gpu-train-rollout branch rather than rewritten.
- tasks.py: DatasetTask, WarmCacheTask, GeometryOracleTask, TrainEpochTask
(one short GPU job per epoch, chained via --resume, which the training
loop already supports unchanged), TrainTask (publishes best.pt/last.pt and
a concatenated metrics.csv so downstream never sees the epoch fan-out),
RolloutTask, AnalysisPrepTask, AnalysisComputeTask (one job per plot x
chunk, walltime sized from run_meta.json at submit time), AnalysisRenderTask
(always local — the only step importing plotstyle/LaTeX), WorkflowTask.
Task bodies call the existing entry points; none of them reimplement
anything.
- run.py + `giant workflow run`: settings wiring and the script b2luigi
re-executes on workers. add_filename_to_cmd is off because b2luigi passes
only the script's basename, and --spec is forwarded via
task_cmd_additional_args so a worker resolves the identical task graph.
configs/workflow_example.toml is the documented starting point.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Groundwork for the b2luigi pipeline orchestration in gitea #83, split out
so the workflow package itself lands as a self-contained change:
- new `workflow` optional-dependency extra (b2luigi, which pulls luigi +
tenacity), included in `dev`.
- deterministic rollout/predict sidecar path: with an explicit `--out`, the
YAML goes to `out.with_suffix(".yaml")` instead of a uuid-named file under
the checkpoint directory, so a workflow task can declare it as a target.
The uuid behaviour is kept for the no-`--out` case, leaving ad-hoc runs and
the /ceph predictions convention untouched.
- epoch-aware shuffle seeding in StreamingStepsDataset (`seed` +
`set_epoch`, the DistributedSampler convention). Shuffling previously drew
from the global numpy state, which `run_train_job` reseeds from
`train.seed` at process start — so a one-epoch-per-job chain would have
replayed the same batch order every epoch. Seeding from
`(seed, epoch, worker_id)` makes epoch k's order identical whether it runs
inside one long `giant train` or as its own resumed job. The val-split
seed is untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLAUDE.md still described the pre-v0.3.0 codebase: the Stage-2
autoregressive redesign as "designed, not implemented", a monolithic
network.py, a single global model.conditioning switch, and WGAN as
"implemented, not yet tested".
- Architecture rewritten around the actual giant/model split
(layers/encoders/trunks/routers/history/objectives/models/builders/
_legacy/summary; network.py is now a re-export shim), plus
cond_layout.py, checkpoint_io.py, _migration.py, data/setup_cache.py
and giant/training/.
- Conditioning documented per axis (conditioning.particle /
conditioning.material, each physical|embedding|onehot, freely mixed).
- Stage 2 documented with both decoders, n_sec.mode, teacher forcing,
stage1_context and the three particle_type.target options.
- Roadmap: v0.3.0 recorded as implemented/released; WGAN and MoE routing
as implemented but unvalidated, with the router retrain as next step.
- Analysis: run dir is <cwd>/analysis_runs/analysis_<id>, plus
variables/reduced/runtime_estimate and analyze list/merge-one/metrics.
- Added giant model summary, configs/, and the CI-automated version and
changelog bump.
README drift fixes only: project tree for the model/analysis/training
splits, analyze run-dir default, missing subcommands, --precision and
--stage2-stage1-context, the extras list, and two accuracy fixes
(--router configures stage 1 only; --conditioning sets two independent
axes at once).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The automated changelog (gitea #50) deliberately started fresh with no
backfill; this reverses that call now that it's wanted. v0.2.0-v0.3.2 are
generated from tag history via git-cliff/cliff.toml, matching the format of
existing entries. v0.3.3 was bumped but never tagged, so its commits stay
folded into the existing v0.3.4 entry. The v0.2.0 range (198 uncurated
pre-automation commits) is hand-curated to drop duplicate commits and
dev-log noise (WIP markers, incomplete-validation runs, repeated
"Apply ruff format").
shower_containment_depth_90/95's title contains a literal "%" (e.g.
"...(90% of deposited energy)"), which usetex reads as a comment marker
and aborts LaTeX compilation. Since render_all processes reduced JSON
files in sorted filename order, this killed every plot id sorting after
these two in the same run.
Escape title/xlabel once, centrally, in render()'s dispatch (the one
place every renderer kind draws them from before handing off to
plotstyle/matplotlib) rather than at each catalog.py call site, so any
future catalog title with a %, &, #, etc. is covered automatically.
_plot_metadata keeps using the unescaped Reduced for the gallery YAML,
since that's not LaTeX.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
giant analyze compares N rollout YAMLs against one shared reference file
(all must name the same dataset, checked up front) instead of exactly one
rollout vs one reference, rendering each rollout as its own colored series
against a single reference line/panel. Series names come from a repeated
--label flag, else the YAML stem, else "rollout" for a single YAML — a
single-rollout run keeps rendering identically to before this change.
Bundle now holds a name-keyed dict of rollout sides instead of one fixed
pair, every catalog compute_partial/finalize builds a Reduced.payload
keyed the same way ("series": {name: ...}, "reference": ... as the one
distinguished non-rollout entry), and every renderer draws N series (or
N panels, for the two heatmap-shaped specs and the router/type-embedding
diagnostics, which are inherently one-matrix/one-checkpoint per rollout)
against the reference's fixed dashed-ink style.
CliRunner stores an uncaught exception in result.exception, not
result.output, so the skip condition never matched and the test
failed outright on CI machines without LaTeX installed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Picks 4 of the 7 catalog additions the issue proposed (the smaller-lift
ones; 2D joint plots, PIT calibration, and the throughput/accuracy scatter
are left for follow-up issues):
- marginal_distance_summary: a var x grouping-axis KS-statistic heatmap,
reusing the existing marginal hist1d compute and just adding a finalize —
a single at-a-glance regression scorecard instead of N overlay plots.
- n_sec_confusion: predicted (rollout) vs true (reference) secondary count
per event, paired by event_id since a rollout is seeded from the same
events as its reference file. Needed a new zero-filling primitive
(reduce.sec_count_by_event) since a plain group_by over secondary rows
silently drops zero-secondary events.
- shower_containment_depth_{90,95}: per-event depth containing 90%/95% of
deposited energy, derived from the same per-event depth-bin matrix the
longitudinal profile already computes.
- router_specialization: max gate weight vs energy per side, summarizing
router_gating's full stacked area into the one trend line the roadmap's
MoE writeup describes (the ~60-65% ceiling), to make a future
lambda_balance>0 retrain's effect on specialization checkable at a glance.
Both new heatmap-shaped plots (distance summary, confusion matrix) share one
new "heatmap" Reduced kind/renderer rather than two near-identical ones.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
lars
gitea-actions