ETPlot/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What This Project Does

A Python package that generates responsive static HTML galleries from scientific plot collections (PDFs and HTMLs). It converts PDFs to PNGs via ImageMagick, organizes plots hierarchically, propagates YAML/JSON metadata through directory trees, and renders everything via a Jinja2 template into a static website served from a web directory.

The repo also ships plotstyle, a standalone matplotlib styling toolkit (KIT corporate-design theme + building-block functions) for producing the PDF figures that feed into a gallery source directory. gallery never imports it — the only connection is the PDF files and metadata.yaml on disk. See plotstyle/CLAUDE.md for full agent-facing usage docs, the API reference, and the metadata.yaml workflow.

Commands

# Install the package (editable)
pip install -e ".[dev]"

# Also install plotstyle's dependency (matplotlib) if working on plot-producing scripts
pip install -e ".[dev,plotting]"   # or: uv sync --extra dev --extra plotting

# Run all tests
pytest tests/

# Run a single test file
pytest tests/test_generate_gallery.py -v
pytest tests/test_plotstyle.py -v

# Run a single test by name
pytest tests/test_generate_gallery.py::test_needs_update_missing_target -v

# Generate gallery
gallery generate --verbose

# Generate with a non-default config
gallery --config config.yaml generate --verbose

# Incremental update for one source only
gallery generate --source /path/to/plots --verbose

# Clean regeneration
gallery generate --clean --verbose

# Launch TUI
gallery tui

# Serve output locally
python -m http.server 8000 -d /web/kschmidt/public_html/

Code style: ruff (lint + format), line-length = 120. Type-checked with ty.

ruff check gallery plotstyle tests
ruff format gallery plotstyle tests
ty check gallery plotstyle
pip-audit

Before committing, run the same checks CI (.gitea/workflows/ci.yml) runs and make sure they pass — ruff check, ruff format --check, ty check, pip-audit, and pytest tests/. Catching a failure locally is faster than waiting on the pipeline.

Architecture

Execution Flow

generate() [api.py]
  └── copy_assets()         [builder.py]  — copies assets/ to web output dir
  └── get_template()        [builder.py]  — loads gallery/templates/gallery.html
  └── build_gallery()       [builder.py]  — recursive per-source-directory walk
        └── load_folder_metadata()  [utils/metadata.py]
        └── merge_metadata()        [utils/metadata.py]  — inherits from parent
        └── process_plot_files()    [utils/processing.py]  — PDF→PNG, copy files
        └── save_metadata_cache()   [utils/metadata.py]
        └── render_gallery_page()   [utils/processing.py]  — writes index.html
        └── recurse into subdirs

Key Files

File	Role
gallery/api.py	generate() — primary public entry point; orchestrates everything
gallery/builder.py	build_gallery() — recursive traversal; get_template(), copy_assets()
gallery/config/__init__.py	GalleryConfig, GallerySource, ConfigManager dataclasses; YAML loading
gallery/cli.py	CLI (gallery command) with subcommands: generate, config, tui, install-completion
gallery/tui.py	TUI (gallery tui) built with Textual; interactive config editor + generation
gallery/utils/processing.py	PDF→PNG via ImageMagick subprocess; needs_update() timestamp check; render_gallery_page()
gallery/utils/metadata.py	Load/merge/cache YAML+JSON metadata; per-plot metadata resolution
gallery/utils/stats.py	Directory size/count statistics
gallery/templates/gallery.html	Single Jinja2 template for all gallery pages
gallery/assets/js/	Vanilla JS modules loaded as ES modules; GalleryApp in gallery-app.js orchestrates all managers
gallery/assets/css/	Modular CSS; main.css imports all others via @import
config.yaml	Local deployment config (paths are machine-specific)
plotstyle/	Standalone matplotlib styling toolkit for producing plots (see plotstyle/CLAUDE.md) — not imported by gallery/
examples/plotstyle_showcase.ipynb	Rendered, runnable tour of every plotstyle function

Config File Format

The YAML config uses a specific structure (not flat — must match GalleryConfig.from_yaml()):

paths:
  web_folder: "/web/user/public_html"   # required
gallery:
  plot_root: "gallery"
  png_dpi: 400
sources:
  - name: "my_plots"
    path: "/path/to/plots"
metadata:
  cache_enabled: true
  inherit_from_parent: true

Incremental Updates

needs_update(source, target) uses a 30-second buffer on mtime comparisons to handle filesystem timing. This is intentional — avoid tightening it.

When source_to_update is passed to generate(), only that source's subdirectory is deleted and rebuilt; all other sources stay intact and the root index is re-rendered to include them.

Metadata Inheritance

metadata.yaml (or .yml/.json) in any source directory is loaded and merged with parent metadata (inherit_from_parent=True by default). Child directories override parent keys. Per-plot overrides can live in <plotname>.yaml files alongside the plot.

Fields are freeform YAML (no fixed schema); title, description, plot_type, experiment get prominent placement in the per-plot metadata popup, everything else still displays under "Additional Information". Text values support inline LaTeX rendered via MathJax client-side.

Plot-Producing Companion (plotstyle)

plotstyle (top-level package, optional plotting extra) is how plots destined for a gallery source directory should be produced — a KIT corporate-design matplotlib theme plus building blocks (new_figure, colorbar, style_legend, panel_label, savefig). It has no code dependency on gallery; the two only meet on disk, via the PDFs and metadata.yaml files a plotstyle script writes into a gallery source directory. Full usage docs, API reference, best practices, and the metadata.yaml workflow live in plotstyle/CLAUDE.md — read that file before writing or reviewing any script that import plotstyle. examples/plotstyle_showcase.ipynb is a rendered, runnable tour of every function.

Frontend (Static JS/CSS)

The frontend is vanilla ES modules — no build step. assets/js/main.js imports GalleryApp from gallery-app.js, which instantiates all manager classes (ThemeManager, SearchManager, NavigationManager, etc.). Each manager is self-contained. The template embeds gallery data as JSON in the page; JS reads it at runtime.

Deployment

The project ships a Dockerfile plus docker-compose.yml (a generator service that runs gallery generate on an interval, and an nginx-based web service serving the output — see deploy/entrypoint.sh and deploy/nginx.conf). This suits a dedicated VM/server you fully control. CI (.gitea/workflows/ci.yml, run via Gitea Actions) runs ruff check, ruff format --check, ty check, pip-audit, and pytest directly against a python:3.11-slim image — no container build/publish in CI. For local development the .venv (or uv) is sufficient.

On shared HPC login nodes without a Docker daemon (e.g. KIT ETP, where public_html is already auto-served) use a plain venv install plus a systemd --user timer instead — see deploy/systemd/README.md.