material is a string literal (e.g. "G4_PbWO4"), not an integer. Store as
object array and key mat_map on str throughout loader and transforms.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- iter_cond_chunks: column-projected row-group streaming; post-step
variables are never read from disk during inference
- build_cond_features: assembles conditioning arrays without any target
or post-step fields
- inv_local_frame_rotation: Rodrigues R^T (negative angle) to rotate
predicted post_dir back from local frame to world frame
- giant predict: loads checkpoint, streams input, runs flow matching
sampler, inverse-normalises and inverse-rotates outputs, writes
predictions incrementally as parquet via PyArrow ParquetWriter
- train now saves model_config in checkpoint so predict can reconstruct
the architecture without extra CLI flags
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Streaming pipeline: row-group-level parquet reading (PyArrow) so
large files never fully land in RAM; Welford online algorithm for
normalizer fitting; StreamingStepsDataset with shuffle buffer and
multi-worker file striping; event-ID scan and vocab scan via cheap
single-column reads
- giant/cli.py: typer-based CLI with `giant train` subcommand, mirroring
scripts/train.py; --shuffle-buffer flag for RAM control
- pyproject.toml: add typer>=0.12 dependency and giant entry point
- train.py: replace len(loader.dataset) with local counters (compatible
with IterableDataset which has no __len__)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lars