Kylian Schmidt
72ebc5e103
✨ Features: - Add comprehensive metadata system with YAML/JSON support - Implement hierarchical metadata inheritance from parent folders - Support plot-specific metadata overrides - Add metadata caching for performance optimization 🗂️ Code Organization: - Move Python orchestration code to orchestration/ folder - Move validation utilities to tests/ folder - Move documentation to docs/ folder - Separate metadata functionality into dedicated module 🔧 Infrastructure: - Add automatic asset copying to web directory - Fix asset path resolution for nested directories - Update template to use dynamic asset paths - Add MetadataConfig class with inheritance options 📚 Documentation: - Add comprehensive metadata usage guide (METADATA_USAGE.md) - Add implementation documentation (METADATA_IMPLEMENTATION.md) - Include example metadata files in examples/ - Add metadata validation utility script 🐛 Bug Fixes: - Fix breadcrumb navigation and JavaScript functionality - Resolve asset path issues in nested directories - Update template imports for modular CSS/JS structure This commit introduces a flexible metadata system that allows users to add rich metadata to plots and folders using YAML or JSON files, with full hierarchical inheritance and plot-specific overrides. The project structure is now better organized with clear separation of concerns.
104 lines
3.5 KiB
Markdown
104 lines
3.5 KiB
Markdown
# Metadata System Implementation Summary
|
|
|
|
## What Was Implemented
|
|
|
|
### 1. Core Metadata Module (`metadata.py`)
|
|
- **`load_metadata_file()`**: Loads YAML/JSON metadata files with error handling
|
|
- **`load_folder_metadata()`**: Discovers and loads folder-level metadata (meta.yaml/meta.json)
|
|
- **`merge_metadata()`**: Merges parent and child metadata with proper override behavior
|
|
- **`resolve_metadata_for_plot()`**: Resolves final metadata for individual plots
|
|
- **`save_metadata_cache()`**: Saves resolved metadata to cache files for performance
|
|
|
|
### 2. Updated Gallery Generator (`generate_gallery.py`)
|
|
- **Hierarchical inheritance**: Folder metadata is inherited by subfolders and plots
|
|
- **Plot-specific overrides**: Individual plots can have their own metadata files
|
|
- **Template integration**: Metadata is passed to HTML templates for rendering
|
|
- **Cache generation**: `meta_cache.json` files are created in each output directory
|
|
|
|
### 3. Configuration Updates (`config.py` and `config.yaml`)
|
|
- Added `MetadataConfig` class with caching and inheritance options
|
|
- Updated main `Config` class to include metadata settings
|
|
- Added metadata section to `config.yaml`
|
|
|
|
### 4. Documentation and Examples
|
|
- **`METADATA_USAGE.md`**: Comprehensive documentation on using the metadata system
|
|
- **`examples/meta.yaml`**: Example folder metadata file
|
|
- **`examples/specific_plot.json`**: Example plot-specific metadata file
|
|
- **`validate_metadata.py`**: Utility script for validating metadata files
|
|
|
|
## Key Features
|
|
|
|
### Hierarchical Metadata Inheritance
|
|
```
|
|
root_folder/
|
|
├── meta.yaml # Base metadata for all plots
|
|
├── subfolder/
|
|
│ ├── meta.yaml # Inherits from parent, can override
|
|
│ ├── plot1.pdf
|
|
│ ├── plot1.yaml # Plot-specific metadata
|
|
│ └── plot2.pdf # Uses folder metadata
|
|
```
|
|
|
|
### Flexible Format Support
|
|
- YAML files: `.yaml`, `.yml`
|
|
- JSON files: `.json`
|
|
- Automatic format detection based on file extension
|
|
|
|
### Template Integration
|
|
- `folder_metadata`: Available in templates for folder-level metadata
|
|
- `item.metadata`: Available for each plot in the items loop
|
|
- Clean separation of concerns between data and presentation
|
|
|
|
### Performance Optimization
|
|
- Metadata caching in `meta_cache.json` files
|
|
- Only reload when source files are newer than cache
|
|
- Efficient hierarchical resolution
|
|
|
|
## Usage Examples
|
|
|
|
### Basic Folder Metadata
|
|
```yaml
|
|
# meta.yaml
|
|
title: "Physics Analysis Results"
|
|
experiment: "CMS"
|
|
author:
|
|
name: "Researcher Name"
|
|
institution: "University"
|
|
tags: ["analysis", "physics"]
|
|
```
|
|
|
|
### Plot-specific Metadata
|
|
```yaml
|
|
# my_plot.yaml (for my_plot.pdf)
|
|
title: "Signal Region Analysis"
|
|
plot_type: "histogram"
|
|
variables:
|
|
x_axis: "mass"
|
|
y_axis: "events"
|
|
highlight: true
|
|
```
|
|
|
|
### Template Usage
|
|
```html
|
|
<h1>{{ folder_metadata.title }}</h1>
|
|
{% for item in items %}
|
|
<div class="plot">
|
|
<h3>{{ item.metadata.title or item.name }}</h3>
|
|
{% if item.metadata.plot_type %}
|
|
<span class="type">{{ item.metadata.plot_type }}</span>
|
|
{% endif %}
|
|
</div>
|
|
{% endfor %}
|
|
```
|
|
|
|
## Benefits
|
|
|
|
1. **Flexibility**: Support any metadata structure using YAML/JSON
|
|
2. **Inheritance**: Avoid repetition by inheriting from parent folders
|
|
3. **Override capability**: Fine-tune metadata for specific plots
|
|
4. **Performance**: Caching system for efficient repeated builds
|
|
5. **Validation**: Built-in error handling and validation utilities
|
|
6. **Documentation**: Comprehensive usage documentation and examples
|
|
|
|
The metadata system is now fully integrated and ready for use in your scientific plot gallery generator!
|