ETPlot/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What This Project Does

A Python package that generates responsive static HTML galleries from scientific plot collections (PDFs and HTMLs). It converts PDFs to PNGs via ImageMagick, organizes plots hierarchically, propagates YAML/JSON metadata through directory trees, and renders everything via a Jinja2 template into a static website served from a web directory.

The repo also ships `plotstyle`, a standalone matplotlib styling toolkit (KIT corporate-design theme + building-block functions) for producing the PDF figures that feed into a `gallery` source directory. `gallery` never imports it — the only connection is the PDF files and `metadata.yaml` on disk. **See `plotstyle/CLAUDE.md` for full agent-facing usage docs, the API reference, and the metadata.yaml workflow.**

This repo is a **uv workspace** (root `pyproject.toml` has `[tool.uv.workspace] members = ["plotstyle"]`) and both projects build with uv's own `uv_build` backend — there is no setuptools anywhere in this repo. Use `uv`/`uv run` for everything; don't reach for `pip install` here.

## Commands

```bash
# Install everything (gallery + plotstyle + dev tools) into the shared workspace venv
uv sync --all-packages

# Run all tests
uv run pytest tests/

# Run a single test file
uv run pytest tests/test_generate_gallery.py -v
uv run pytest tests/test_plotstyle.py -v

# Run a single test by name
uv run pytest tests/test_generate_gallery.py::test_needs_update_missing_target -v

# Generate gallery
uv run gallery generate --verbose

# Generate with a non-default config
uv run gallery --config config.yaml generate --verbose

# Incremental update for one source only
uv run gallery generate --source /path/to/plots --verbose

# Clean regeneration
uv run gallery generate --clean --verbose

# Launch TUI
uv run gallery tui

# Serve output locally
python -m http.server 8000 -d /web/kschmidt/public_html/
```

Code style: ruff (lint + format), `line-length = 120`. Type-checked with `ty`.

```bash
uv run ruff check gallery plotstyle tests
uv run ruff format gallery plotstyle tests
uv run ty check gallery plotstyle
uv run pip-audit --skip-editable
```

**Before committing**, run the same checks CI (`.gitea/workflows/ci.yml`) runs and make sure they pass — `ruff check`, `ruff format --check`, `ty check`, `pip-audit`, and `pytest tests/`. Catching a failure locally is faster than waiting on the pipeline.

## Architecture

### Execution Flow

```
generate() [api.py]
  └── copy_assets()         [builder.py]  — copies assets/ to web output dir
  └── get_template()        [builder.py]  — loads gallery/templates/gallery.html
  └── build_gallery()       [builder.py]  — recursive per-source-directory walk
        └── load_folder_metadata()  [utils/metadata.py]
        └── merge_metadata()        [utils/metadata.py]  — inherits from parent
        └── process_plot_files()    [utils/processing.py]  — PDF→PNG, copy files
        └── save_metadata_cache()   [utils/metadata.py]
        └── render_gallery_page()   [utils/processing.py]  — writes index.html
        └── recurse into subdirs
```

### Key Files

| File | Role |
|------|------|
| `gallery/api.py` | `generate()` — primary public entry point; orchestrates everything |
| `gallery/builder.py` | `build_gallery()` — recursive traversal; `get_template()`, `copy_assets()` |
| `gallery/config/__init__.py` | `GalleryConfig`, `GallerySource`, `ConfigManager` dataclasses; YAML loading |
| `gallery/cli.py` | CLI (`gallery` command) with subcommands: `generate`, `config`, `tui`, `install-completion` |
| `gallery/tui.py` | TUI (`gallery tui`) built with Textual; interactive config editor + generation |
| `gallery/utils/processing.py` | PDF→PNG via ImageMagick subprocess; `needs_update()` timestamp check; `render_gallery_page()` |
| `gallery/utils/metadata.py` | Load/merge/cache YAML+JSON metadata; per-plot metadata resolution |
| `gallery/utils/stats.py` | Directory size/count statistics |
| `gallery/templates/gallery.html` | Single Jinja2 template for all gallery pages |
| `gallery/assets/js/` | Vanilla JS modules loaded as ES modules; `GalleryApp` in `gallery-app.js` orchestrates all managers |
| `gallery/assets/css/` | Modular CSS; `main.css` imports all others via `@import` |
| `config.yaml` | Local deployment config (paths are machine-specific) |
| `plotstyle/` | Standalone matplotlib styling toolkit for producing plots (see `plotstyle/CLAUDE.md`) — not imported by `gallery/`; a separate uv workspace member with its own `pyproject.toml`, code under `plotstyle/src/plotstyle/` (src layout) |
| `.gitea/workflows/publish-plotstyle.yml` | Builds and publishes `plotstyle` to the Gitea package registry on `plotstyle-v*` tags |
| `examples/plotstyle_showcase.ipynb` | Rendered, runnable tour of every `plotstyle` function |

### Config File Format

The YAML config uses a specific structure (not flat — must match `GalleryConfig.from_yaml()`):

```yaml
paths:
  web_folder: "/web/user/public_html"   # required
gallery:
  plot_root: "gallery"
  png_dpi: 400
sources:
  - name: "my_plots"
    path: "/path/to/plots"
metadata:
  cache_enabled: true
  inherit_from_parent: true
```

### Incremental Updates

`needs_update(source, target)` uses a **30-second buffer** on mtime comparisons to handle filesystem timing. This is intentional — avoid tightening it.

When `source_to_update` is passed to `generate()`, only that source's subdirectory is deleted and rebuilt; all other sources stay intact and the root index is re-rendered to include them.

### Metadata Inheritance

`metadata.yaml` (or `.yml`/`.json`) in any source directory is loaded and **merged with parent metadata** (`inherit_from_parent=True` by default). Child directories override parent keys. Per-plot overrides can live in `<plotname>.yaml` files alongside the plot.

Fields are freeform YAML (no fixed schema); `title`, `description`, `plot_type`, `experiment` get prominent placement in the per-plot metadata popup, everything else still displays under "Additional Information". Text values support inline LaTeX rendered via MathJax client-side.

### Plot-Producing Companion (`plotstyle`)

`plotstyle` (a separate uv workspace member, see below) is how plots destined for a `gallery` source directory should be produced — a KIT corporate-design matplotlib theme plus building blocks (`new_figure`, `colorbar`, `style_legend`, `panel_label`, `savefig`). It has no code dependency on `gallery`; the two only meet on disk, via the PDFs and `metadata.yaml` files a `plotstyle` script writes into a `gallery` source directory. **Full usage docs, API reference, best practices, and the metadata.yaml workflow live in `plotstyle/CLAUDE.md`** — read that file before writing or reviewing any script that `import plotstyle`. `examples/plotstyle_showcase.ipynb` is a rendered, runnable tour of every function.

`plotstyle` has its own `plotstyle/pyproject.toml` (own `uv_build` project, code lives in `plotstyle/src/plotstyle/`) and is a member of this repo's uv workspace (`[tool.uv.workspace] members = ["plotstyle"]` in the root `pyproject.toml`) — `uv sync --all-packages` installs both `gallery` and `plotstyle` (and matplotlib) into one shared venv, which is why `import plotstyle` works from `tests/` and the example notebook without `gallery` ever depending on it. It's also independently publishable: pushing a tag matching `plotstyle-v*` (e.g. `plotstyle-v0.1.0`) runs `.gitea/workflows/publish-plotstyle.yml`, which runs `uv build --package plotstyle` and `uv publish` to this repo's Gitea package registry (`{server}/api/packages/{owner}/pypi`) using the workflow's auto-generated token. The tag's version suffix must match `plotstyle/pyproject.toml`'s `version` field or the workflow fails fast — bump that version before tagging a new release.

### Frontend (Static JS/CSS)

The frontend is vanilla ES modules — no build step. `assets/js/main.js` imports `GalleryApp` from `gallery-app.js`, which instantiates all manager classes (`ThemeManager`, `SearchManager`, `NavigationManager`, etc.). Each manager is self-contained. The template embeds gallery data as JSON in the page; JS reads it at runtime.

### Deployment

The project ships a `Dockerfile` plus `docker-compose.yml` (a `generator` service that runs `gallery generate` on an interval, and an `nginx`-based `web` service serving the output — see `deploy/entrypoint.sh` and `deploy/nginx.conf`). This suits a dedicated VM/server you fully control. The `Dockerfile` is a two-stage build: a builder stage copies in the official `ghcr.io/astral-sh/uv` binary and runs `uv build --package gallery` to produce a wheel, then the final stage `pip install`s just that wheel (no dev deps, no workspace/lockfile needed at runtime). CI (`.gitea/workflows/ci.yml`, run via Gitea Actions) uses `astral-sh/setup-uv` + `uv sync --all-packages` and runs every check (`ruff check`, `ruff format --check`, `ty check`, `pip-audit`, `pytest`) via `uv run` — no plain `pip`/`actions/setup-python` anywhere in CI. For local development, `uv sync --all-packages` is all you need.

On shared HPC login nodes without a Docker daemon (e.g. KIT ETP, where `public_html` is already auto-served) use a plain venv install plus a `systemd --user` timer instead — see `deploy/systemd/README.md`.