ETPlot/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What This Project Does

A Python package that generates responsive static HTML galleries from scientific plot collections (PDFs and HTMLs). It converts PDFs to PNGs via ImageMagick, organizes plots hierarchically, propagates YAML/JSON metadata through directory trees, and renders everything via a Jinja2 template into a static website served from a web directory.

The repo also ships `plotstyle`, a standalone matplotlib styling toolkit (KIT corporate-design theme + building-block functions) for producing the PDF figures that feed into a `gallery` source directory. `gallery` never imports it — the only connection is the PDF files and `metadata.yaml` on disk. **See `plotstyle/CLAUDE.md` for full agent-facing usage docs, the API reference, and the metadata.yaml workflow.**

## Commands

```bash
# Install the package (editable)
pip install -e ".[dev]"

# Also install plotstyle's dependency (matplotlib) if working on plot-producing scripts
pip install -e ".[dev,plotting]"   # or: uv sync --extra dev --extra plotting

# Run all tests
pytest tests/

# Run a single test file
pytest tests/test_generate_gallery.py -v
pytest tests/test_plotstyle.py -v

# Run a single test by name
pytest tests/test_generate_gallery.py::test_needs_update_missing_target -v

# Generate gallery
gallery generate --verbose

# Generate with a non-default config
gallery --config config.yaml generate --verbose

# Incremental update for one source only
gallery generate --source /path/to/plots --verbose

# Clean regeneration
gallery generate --clean --verbose

# Launch TUI
gallery tui

# Serve output locally
python -m http.server 8000 -d /web/kschmidt/public_html/
```

Code style: ruff (lint + format), `line-length = 120`. Type-checked with `ty`.

```bash
ruff check gallery plotstyle tests
ruff format gallery plotstyle tests
ty check gallery plotstyle
pip-audit
```

**Before committing**, run the same checks CI (`.gitea/workflows/ci.yml`) runs and make sure they pass — `ruff check`, `ruff format --check`, `ty check`, `pip-audit`, and `pytest tests/`. Catching a failure locally is faster than waiting on the pipeline.

## Architecture

### Execution Flow

```
generate() [api.py]
  └── copy_assets()         [builder.py]  — copies assets/ to web output dir
  └── get_template()        [builder.py]  — loads gallery/templates/gallery.html
  └── build_gallery()       [builder.py]  — recursive per-source-directory walk
        └── load_folder_metadata()  [utils/metadata.py]
        └── merge_metadata()        [utils/metadata.py]  — inherits from parent
        └── process_plot_files()    [utils/processing.py]  — PDF→PNG, copy files
        └── save_metadata_cache()   [utils/metadata.py]
        └── render_gallery_page()   [utils/processing.py]  — writes index.html
        └── recurse into subdirs
```

### Key Files

| File | Role |
|------|------|
| `gallery/api.py` | `generate()` — primary public entry point; orchestrates everything |
| `gallery/builder.py` | `build_gallery()` — recursive traversal; `get_template()`, `copy_assets()` |
| `gallery/config/__init__.py` | `GalleryConfig`, `GallerySource`, `ConfigManager` dataclasses; YAML loading |
| `gallery/cli.py` | CLI (`gallery` command) with subcommands: `generate`, `config`, `tui`, `install-completion` |
| `gallery/tui.py` | TUI (`gallery tui`) built with Textual; interactive config editor + generation |
| `gallery/utils/processing.py` | PDF→PNG via ImageMagick subprocess; `needs_update()` timestamp check; `render_gallery_page()` |
| `gallery/utils/metadata.py` | Load/merge/cache YAML+JSON metadata; per-plot metadata resolution |
| `gallery/utils/stats.py` | Directory size/count statistics |
| `gallery/templates/gallery.html` | Single Jinja2 template for all gallery pages |
| `gallery/assets/js/` | Vanilla JS modules loaded as ES modules; `GalleryApp` in `gallery-app.js` orchestrates all managers |
| `gallery/assets/css/` | Modular CSS; `main.css` imports all others via `@import` |
| `config.yaml` | Local deployment config (paths are machine-specific) |
| `plotstyle/` | Standalone matplotlib styling toolkit for producing plots (see `plotstyle/CLAUDE.md`) — not imported by `gallery/` |
| `examples/plotstyle_showcase.ipynb` | Rendered, runnable tour of every `plotstyle` function |

### Config File Format

The YAML config uses a specific structure (not flat — must match `GalleryConfig.from_yaml()`):

```yaml
paths:
  web_folder: "/web/user/public_html"   # required
gallery:
  plot_root: "gallery"
  png_dpi: 400
sources:
  - name: "my_plots"
    path: "/path/to/plots"
metadata:
  cache_enabled: true
  inherit_from_parent: true
```

### Incremental Updates

`needs_update(source, target)` uses a **30-second buffer** on mtime comparisons to handle filesystem timing. This is intentional — avoid tightening it.

When `source_to_update` is passed to `generate()`, only that source's subdirectory is deleted and rebuilt; all other sources stay intact and the root index is re-rendered to include them.

### Metadata Inheritance

`metadata.yaml` (or `.yml`/`.json`) in any source directory is loaded and **merged with parent metadata** (`inherit_from_parent=True` by default). Child directories override parent keys. Per-plot overrides can live in `<plotname>.yaml` files alongside the plot.

Fields are freeform YAML (no fixed schema); `title`, `description`, `plot_type`, `experiment` get prominent placement in the per-plot metadata popup, everything else still displays under "Additional Information". Text values support inline LaTeX rendered via MathJax client-side.

### Plot-Producing Companion (`plotstyle`)

`plotstyle` (top-level package, optional `plotting` extra) is how plots destined for a `gallery` source directory should be produced — a KIT corporate-design matplotlib theme plus building blocks (`new_figure`, `colorbar`, `style_legend`, `panel_label`, `savefig`). It has no code dependency on `gallery`; the two only meet on disk, via the PDFs and `metadata.yaml` files a `plotstyle` script writes into a `gallery` source directory. **Full usage docs, API reference, best practices, and the metadata.yaml workflow live in `plotstyle/CLAUDE.md`** — read that file before writing or reviewing any script that `import plotstyle`. `examples/plotstyle_showcase.ipynb` is a rendered, runnable tour of every function.

### Frontend (Static JS/CSS)

The frontend is vanilla ES modules — no build step. `assets/js/main.js` imports `GalleryApp` from `gallery-app.js`, which instantiates all manager classes (`ThemeManager`, `SearchManager`, `NavigationManager`, etc.). Each manager is self-contained. The template embeds gallery data as JSON in the page; JS reads it at runtime.

### Deployment

The project ships a `Dockerfile` plus `docker-compose.yml` (a `generator` service that runs `gallery generate` on an interval, and an `nginx`-based `web` service serving the output — see `deploy/entrypoint.sh` and `deploy/nginx.conf`). This suits a dedicated VM/server you fully control. CI (`.gitea/workflows/ci.yml`, run via Gitea Actions) runs `ruff check`, `ruff format --check`, `ty check`, `pip-audit`, and pytest directly against a `python:3.11-slim` image — no container build/publish in CI. For local development the `.venv` (or `uv`) is sufficient.

On shared HPC login nodes without a Docker daemon (e.g. KIT ETP, where `public_html` is already auto-served) use a plain venv install plus a `systemd --user` timer instead — see `deploy/systemd/README.md`.