ETPlot/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What This Project Does

A Python package that generates responsive static HTML galleries from scientific plot collections (PDFs and HTMLs). It converts PDFs to PNGs via ImageMagick, organizes plots hierarchically, propagates YAML/JSON metadata through directory trees, and renders everything via a Jinja2 template into a static website served from a web directory.

The repo also ships plotstyle, a standalone matplotlib styling toolkit (KIT corporate-design theme + building-block functions) for producing the PDF figures that feed into a gallery source directory. gallery never imports it — the only connection is the PDF files and metadata.yaml on disk. See plotstyle/CLAUDE.md for full agent-facing usage docs, the API reference, and the metadata.yaml workflow.

This repo is a uv workspace (root pyproject.toml has [tool.uv.workspace] members = ["plotstyle"]) and both projects build with uv's own uv_build backend — there is no setuptools anywhere in this repo. Use uv/uv run for everything; don't reach for pip install here.

Commands

# Install everything (gallery + plotstyle + dev tools) into the shared workspace venv
uv sync --all-packages

# Run all tests
uv run pytest tests/

# Run a single test file
uv run pytest tests/test_generate_gallery.py -v
uv run pytest tests/test_plotstyle.py -v

# Run a single test by name
uv run pytest tests/test_generate_gallery.py::test_needs_update_missing_target -v

# Generate gallery
uv run gallery generate --verbose

# Generate with a non-default config
uv run gallery --config config.yaml generate --verbose

# Incremental update for one source only
uv run gallery generate --source /path/to/plots --verbose

# Clean regeneration
uv run gallery generate --clean --verbose

# Launch TUI
uv run gallery tui

# Serve output locally
python -m http.server 8000 -d /web/kschmidt/public_html/

Code style: ruff (lint + format), line-length = 120. Type-checked with ty.

uv run ruff check gallery plotstyle tests
uv run ruff format gallery plotstyle tests
uv run ty check gallery plotstyle
uv run pip-audit --skip-editable

Before committing, run the same checks CI (.gitea/workflows/ci.yml) runs and make sure they pass — ruff check, ruff format --check, ty check, pip-audit, and pytest tests/. Catching a failure locally is faster than waiting on the pipeline.

Architecture

Execution Flow

generate() [api.py]
  └── copy_assets()         [builder.py]  — copies assets/ to web output dir
  └── get_template()        [builder.py]  — loads gallery/templates/gallery.html
  └── build_gallery()       [builder.py]  — recursive per-source-directory walk
        └── load_folder_metadata()  [utils/metadata.py]
        └── merge_metadata()        [utils/metadata.py]  — inherits from parent
        └── process_plot_files()    [utils/processing.py]  — PDF→PNG, copy files
        └── save_metadata_cache()   [utils/metadata.py]
        └── render_gallery_page()   [utils/processing.py]  — writes index.html
        └── recurse into subdirs

Key Files

File	Role
gallery/api.py	generate() — primary public entry point; orchestrates everything
gallery/builder.py	build_gallery() — recursive traversal; get_template(), copy_assets()
gallery/config/__init__.py	GalleryConfig, GallerySource, ConfigManager dataclasses; YAML loading
gallery/cli.py	CLI (gallery command) with subcommands: generate, config, tui, install-completion
gallery/tui.py	TUI (gallery tui) built with Textual; interactive config editor + generation
gallery/utils/processing.py	PDF→PNG via ImageMagick subprocess; needs_update() timestamp check; render_gallery_page()
gallery/utils/metadata.py	Load/merge/cache YAML+JSON metadata; per-plot metadata resolution
gallery/utils/stats.py	Directory size/count statistics
gallery/templates/gallery.html	Single Jinja2 template for all gallery pages
gallery/assets/js/	Vanilla JS modules loaded as ES modules; GalleryApp in gallery-app.js orchestrates all managers
gallery/assets/css/	Modular CSS; main.css imports all others via @import
config.yaml	Local deployment config (paths are machine-specific)
plotstyle/	Standalone matplotlib styling toolkit for producing plots (see plotstyle/CLAUDE.md) — not imported by gallery/; a separate uv workspace member with its own pyproject.toml, code under plotstyle/src/plotstyle/ (src layout)
.gitea/workflows/publish-plotstyle.yml	Builds and publishes plotstyle to the Gitea package registry on plotstyle-v* tags
examples/plotstyle_showcase.ipynb	Rendered, runnable tour of every plotstyle function

Config File Format

The YAML config uses a specific structure (not flat — must match GalleryConfig.from_yaml()):

paths:
  web_folder: "/web/user/public_html"   # required
gallery:
  plot_root: "gallery"
  png_dpi: 400
sources:
  - name: "my_plots"
    path: "/path/to/plots"
metadata:
  cache_enabled: true
  inherit_from_parent: true

Incremental Updates

needs_update(source, target) uses a 30-second buffer on mtime comparisons to handle filesystem timing. This is intentional — avoid tightening it.

When source_to_update is passed to generate(), only that source's subdirectory is deleted and rebuilt; all other sources stay intact and the root index is re-rendered to include them.

Metadata Inheritance

metadata.yaml (or .yml/.json) in any source directory is loaded and merged with parent metadata (inherit_from_parent=True by default). Child directories override parent keys. Per-plot overrides can live in <plotname>.yaml files alongside the plot.

Fields are freeform YAML (no fixed schema); title, description, plot_type, experiment get prominent placement in the per-plot metadata popup, everything else still displays under "Additional Information". Text values support inline LaTeX rendered via MathJax client-side.

Plot-Producing Companion (plotstyle)

plotstyle (a separate uv workspace member, see below) is how plots destined for a gallery source directory should be produced — a KIT corporate-design matplotlib theme plus building blocks (new_figure, colorbar, style_legend, panel_label, savefig). It has no code dependency on gallery; the two only meet on disk, via the PDFs and metadata.yaml files a plotstyle script writes into a gallery source directory. Full usage docs, API reference, best practices, and the metadata.yaml workflow live in plotstyle/CLAUDE.md — read that file before writing or reviewing any script that import plotstyle. examples/plotstyle_showcase.ipynb is a rendered, runnable tour of every function.

plotstyle has its own plotstyle/pyproject.toml (own uv_build project, code lives in plotstyle/src/plotstyle/) and is a member of this repo's uv workspace ([tool.uv.workspace] members = ["plotstyle"] in the root pyproject.toml) — uv sync --all-packages installs both gallery and plotstyle (and matplotlib) into one shared venv, which is why import plotstyle works from tests/ and the example notebook without gallery ever depending on it. It's also independently publishable: pushing a tag matching plotstyle-v* (e.g. plotstyle-v0.1.0) runs .gitea/workflows/publish-plotstyle.yml, which runs uv build --package plotstyle and uv publish to this repo's Gitea package registry ({server}/api/packages/{owner}/pypi) using the workflow's auto-generated token. The tag's version suffix must match plotstyle/pyproject.toml's version field or the workflow fails fast — bump that version before tagging a new release.

Frontend (Static JS/CSS)

The frontend is vanilla ES modules — no build step. assets/js/main.js imports GalleryApp from gallery-app.js, which instantiates all manager classes (ThemeManager, SearchManager, NavigationManager, etc.). Each manager is self-contained. The template embeds gallery data as JSON in the page; JS reads it at runtime.

Deployment

The project ships a Dockerfile plus docker-compose.yml (a generator service that runs gallery generate on an interval, and an nginx-based web service serving the output — see deploy/entrypoint.sh and deploy/nginx.conf). This suits a dedicated VM/server you fully control. The Dockerfile is a two-stage build: a builder stage copies in the official ghcr.io/astral-sh/uv binary and runs uv build --package gallery to produce a wheel, then the final stage pip installs just that wheel (no dev deps, no workspace/lockfile needed at runtime). CI (.gitea/workflows/ci.yml, run via Gitea Actions) uses astral-sh/setup-uv + uv sync --all-packages and runs every check (ruff check, ruff format --check, ty check, pip-audit, pytest) via uv run — no plain pip/actions/setup-python anywhere in CI. For local development, uv sync --all-packages is all you need.

On shared HPC login nodes without a Docker daemon (e.g. KIT ETP, where public_html is already auto-served) use a plain venv install plus a systemd --user timer instead — see deploy/systemd/README.md.