Remove the hand-rolled analysis submit path (gitea #83)
b2luigi's AnalysisComputeTask now submits the per-(plot, chunk) jobs, so the bespoke submit-file generator has nothing left to do: - giant/analysis/condor.py -> giant/analysis/run.py, dropping SubmitConfig, the wrapper/submit-description templates, _job_walltimes and _resolve_giant_executable. What stays is the actual logic — prep, RunMeta, the rollout-YAML loading, compute_reduced/compute_one and merge_one/merge_all — and the module no longer submits anything, hence the name. - `giant analyze submit` is gone; prep / compute-one / merge-one / list / render / metrics remain as the single-step primitives the workflow calls. - tests/test_condor.py -> tests/test_analysis_run.py, minus the submit-description cases. CLAUDE.md and README.md document the workflow package, the new `workflow` extra, and — for whenever condor-gpu-train-rollout is merged — that its train-submit/rollout-submit commands are deliberately superseded and must not be revived. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -110,10 +110,15 @@ giant/
|
||||
│ │ ├── reduced.py # Partial/Reduced — the compact JSON a compute job emits
|
||||
│ │ ├── catalog.py # declarative PlotSpec registry (`giant analyze list`)
|
||||
│ │ ├── router_gating.py / type_embedding_distance.py # checkpoint-bound diagnostics
|
||||
│ │ ├── runtime_estimate.py # per-(plot, chunk) walltime estimates for submit
|
||||
│ │ ├── condor.py # prep / compute-one / merge / submit-description plumbing
|
||||
│ │ ├── runtime_estimate.py # per-(plot, chunk) walltime estimates for the job requests
|
||||
│ │ ├── run.py # prep / compute-one / merge plumbing
|
||||
│ │ └── render.py # PDFs + HTML gallery (only module importing plotstyle/LaTeX)
|
||||
│ └── cli.py # `giant train` / `new-run` / `model summary` / `predict` / `rollout` / `analyze`
|
||||
│ ├── workflow/ # b2luigi pipeline orchestration (`giant workflow run spec.toml`)
|
||||
│ │ ├── spec.py # workflow TOML -> frozen dataclasses, validation, per-task spec hashes
|
||||
│ │ ├── htcondor.py # CPU/GPU submit settings (docker image, +RemoteJob, GPU requirements)
|
||||
│ │ ├── tasks.py # the task graph: cache-warm -> train (one job/epoch) -> rollout -> analysis
|
||||
│ │ └── run.py # the script b2luigi re-executes on every worker
|
||||
│ └── cli.py # `giant train` / `new-run` / `model summary` / `predict` / `rollout` / `analyze` / `workflow`
|
||||
├── giant/tools/ # dataset/tooling logic, unified under the `dwarf` CLI (`dwarf --help`)
|
||||
│ ├── dwarf.py # Typer app: convert, migrate, bump-gen, bump-schema, status,
|
||||
│ │ # update-manifest, create-manifest, make-root,
|
||||
@@ -140,9 +145,10 @@ uv sync --extra cpu --extra geometry # add scikit-learn, for `dwarf build-geome
|
||||
uv sync --extra cpu --extra analysis # matplotlib/polars/plotstyle, for `giant analyze render`
|
||||
uv sync --extra cpu --extra convert # uproot/awkward/polars, for `dwarf convert`
|
||||
uv sync --extra cpu --extra wandb # W&B logging (`giant train --wandb`)
|
||||
uv sync --extra cpu --extra workflow # b2luigi, for `giant workflow run`
|
||||
```
|
||||
|
||||
The `dev` extra pulls in `convert`, `analysis`, `geometry` and `wandb` as well.
|
||||
The `dev` extra pulls in `convert`, `analysis`, `geometry`, `wandb` and `workflow` as well.
|
||||
|
||||
`cpu` and `cuda` are mutually exclusive — pick one to select the torch build (pinned to 2.3.x). Plain `uv sync` installs no torch at all. See `CLAUDE.md` for details.
|
||||
|
||||
@@ -179,17 +185,29 @@ Config-file-only knobs (no CLI flag — use `--config config.toml`): `stage2_mod
|
||||
- `giant analyze` — deeper rollout-vs-reference diagnostics (marginals by energy/pdg/material, per-event totals, shower profiles, species share, leakage, secondaries):
|
||||
|
||||
```bash
|
||||
giant analyze submit rollout.yaml --accounting-group cms # prep + one HTCondor job per plot × chunk (compute only)
|
||||
giant analyze submit a.yaml b.yaml --accounting-group cms --label flow --label wgan # N rollouts vs one shared reference
|
||||
giant analyze prep rollout.yaml --chunks 8 # lay out the run directory
|
||||
giant analyze prep a.yaml b.yaml --label flow --label wgan # N rollouts vs one shared reference
|
||||
giant analyze render <run_dir> --gallery # local: merge chunks, then styled PDFs + HTML gallery (needs LaTeX)
|
||||
|
||||
giant analyze list # every catalog plot id
|
||||
giant analyze prep rollout.yaml --chunks 8 # just the run directory, no submission
|
||||
giant analyze compute-one --id marginal_edep --run-dir <run_dir> --chunk 0 # what a condor job runs
|
||||
giant analyze merge-one --id marginal_edep --run-dir <run_dir> # merge one plot's chunks (debugging)
|
||||
```
|
||||
|
||||
`<run_dir>` defaults to `<cwd>/analysis_runs/analysis_<id>` (`--run-dir` overrides it; `prep`/`submit` print it). Multiple rollout YAMLs must all name the same reference (`dataset`) file; each renders as its own colored series against one reference line/panel. Compute jobs are polars/numpy only; only `render` needs LaTeX, so it always runs locally.
|
||||
The per-(plot, chunk) compute jobs themselves are submitted by the workflow (below), not by `giant analyze` — these commands are the single-step primitives it calls. `<run_dir>` defaults to `<cwd>/analysis_runs/analysis_<id>` (`--run-dir` overrides it; `prep` prints it). Multiple rollout YAMLs must all name the same reference (`dataset`) file; each renders as its own colored series against one reference line/panel. Compute jobs are polars/numpy only; only `render` needs LaTeX, so it always runs locally.
|
||||
|
||||
## Workflow orchestration
|
||||
|
||||
Multi-step pipelines run through [b2luigi](https://github.com/belle2/b2luigi) — one spec file describes a whole experiment, and every step's outputs are files on `/ceph` that are only recomputed when their spec (or an upstream one) changes:
|
||||
|
||||
```bash
|
||||
uv sync --extra cpu --extra workflow
|
||||
giant workflow run configs/workflow_example.toml --mode dry-run # what would run
|
||||
giant workflow run configs/workflow_example.toml --mode show-output # where every output goes
|
||||
giant workflow run configs/workflow_example.toml --batch --workers 20 # submit to HTCondor and wait
|
||||
```
|
||||
|
||||
The spec holds `[workflow]`/`[condor]`/`[dataset]`/`[geometry]` plus repeated `[[train]]`, `[[rollout]]` and `[[analysis]]` tables cross-referenced by name (see `configs/workflow_example.toml`). The task graph is `DatasetTask → WarmCacheTask/GeometryOracleTask → TrainEpochTask… → TrainTask → RolloutTask → AnalysisPrepTask → AnalysisComputeTask(plot, chunk) → AnalysisRenderTask`. Training is split into one short GPU job per epoch (chained by `--resume`), which schedules better on a busy farm and survives preemption; `TrainTask` then publishes one `best.pt`/`last.pt`/`metrics.csv` for everything downstream. Rendering always runs locally, since it is the only step that needs LaTeX.
|
||||
|
||||
Separately, `giant analyze metrics <train_run_dir>` renders training-progress plots (loss/lr/accuracy/grad-norm/router/wgan/throughput) straight from a training run's `metrics.csv`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user