Added so much stuff I cant keep up. I think some sorting capability and a way to add metadata to each folder (WIP)
This commit is contained in:
Kylian Schmidt
@@ -1,101 +0,0 @@
|
||||
# PDF Export Functionality
|
||||
|
||||
The gallery now includes the ability to export multiple plots to a merged PDF with grid layout.
|
||||
|
||||
## Features
|
||||
|
||||
- Select up to 4 plots from any gallery page
|
||||
- Automatic grid layout (1x1, 1x2, 2x2)
|
||||
- Export to PDF with proper scaling
|
||||
- Keyboard shortcuts for easy access
|
||||
|
||||
## Usage
|
||||
|
||||
### Selecting Plots
|
||||
|
||||
1. Press **Ctrl+E** to enter selection mode
|
||||
2. Click on up to 4 plot thumbnails to select them
|
||||
3. Selected plots will show a checkmark overlay
|
||||
4. A counter shows how many plots are selected (e.g., "2/4 selected")
|
||||
|
||||
### Exporting
|
||||
|
||||
1. After selecting plots, click the **📄** export button (appears in floating buttons)
|
||||
2. The system will show export instructions with:
|
||||
- JSON data for the export request
|
||||
- Command to run the export script
|
||||
3. Copy the JSON data and save it as `export_request.json`
|
||||
4. Run the export command in your terminal
|
||||
|
||||
### Keyboard Shortcuts
|
||||
|
||||
- **Ctrl+E**: Toggle selection mode
|
||||
- **Escape**: Clear all selections and exit selection mode
|
||||
|
||||
## Export Methods
|
||||
|
||||
The system supports two PDF merging methods:
|
||||
|
||||
### Method 1: pdfjam (Recommended)
|
||||
```bash
|
||||
# Install on Ubuntu/Debian
|
||||
sudo apt-get install texlive-extra-utils
|
||||
|
||||
# Check if available
|
||||
python export_plots.py --check-deps
|
||||
```
|
||||
|
||||
### Method 2: Python libraries
|
||||
```bash
|
||||
# Install Python dependencies
|
||||
pip install PyPDF2 reportlab
|
||||
|
||||
# Check if available
|
||||
python export_plots.py --check-deps
|
||||
```
|
||||
|
||||
## Export Script Usage
|
||||
|
||||
```bash
|
||||
# Basic usage
|
||||
python export_plots.py export_request.json
|
||||
|
||||
# Specify output file
|
||||
python export_plots.py export_request.json --output my_plots.pdf
|
||||
|
||||
# Check available dependencies
|
||||
python export_plots.py --check-deps
|
||||
```
|
||||
|
||||
## JSON Request Format
|
||||
|
||||
```json
|
||||
{
|
||||
"plots": [
|
||||
"/path/to/plot1.pdf",
|
||||
"/path/to/plot2.pdf"
|
||||
],
|
||||
"layout": {
|
||||
"rows": 1,
|
||||
"cols": 2
|
||||
},
|
||||
"output_name": "merged_plots.pdf"
|
||||
}
|
||||
```
|
||||
|
||||
## Layout Options
|
||||
|
||||
- **1 plot**: 1x1 grid
|
||||
- **2 plots**: 1x2 grid (horizontal)
|
||||
- **3 plots**: 2x2 grid (one empty slot)
|
||||
- **4 plots**: 2x2 grid (full)
|
||||
|
||||
## Technical Details
|
||||
|
||||
The export functionality consists of:
|
||||
|
||||
- **Frontend**: JavaScript selection UI and export manager
|
||||
- **Backend**: Python script for PDF merging
|
||||
- **CSS**: Styling for selection mode and overlays
|
||||
|
||||
The system is designed to work without requiring a web server, using file-based communication between the browser and Python script.
|
||||
@@ -1,103 +0,0 @@
|
||||
# Metadata System Implementation Summary
|
||||
|
||||
## What Was Implemented
|
||||
|
||||
### 1. Core Metadata Module (`metadata.py`)
|
||||
- **`load_metadata_file()`**: Loads YAML/JSON metadata files with error handling
|
||||
- **`load_folder_metadata()`**: Discovers and loads folder-level metadata (meta.yaml/meta.json)
|
||||
- **`merge_metadata()`**: Merges parent and child metadata with proper override behavior
|
||||
- **`resolve_metadata_for_plot()`**: Resolves final metadata for individual plots
|
||||
- **`save_metadata_cache()`**: Saves resolved metadata to cache files for performance
|
||||
|
||||
### 2. Updated Gallery Generator (`generate_gallery.py`)
|
||||
- **Hierarchical inheritance**: Folder metadata is inherited by subfolders and plots
|
||||
- **Plot-specific overrides**: Individual plots can have their own metadata files
|
||||
- **Template integration**: Metadata is passed to HTML templates for rendering
|
||||
- **Cache generation**: `meta_cache.json` files are created in each output directory
|
||||
|
||||
### 3. Configuration Updates (`config.py` and `config.yaml`)
|
||||
- Added `MetadataConfig` class with caching and inheritance options
|
||||
- Updated main `Config` class to include metadata settings
|
||||
- Added metadata section to `config.yaml`
|
||||
|
||||
### 4. Documentation and Examples
|
||||
- **`METADATA_USAGE.md`**: Comprehensive documentation on using the metadata system
|
||||
- **`examples/meta.yaml`**: Example folder metadata file
|
||||
- **`examples/specific_plot.json`**: Example plot-specific metadata file
|
||||
- **`validate_metadata.py`**: Utility script for validating metadata files
|
||||
|
||||
## Key Features
|
||||
|
||||
### Hierarchical Metadata Inheritance
|
||||
```
|
||||
root_folder/
|
||||
├── meta.yaml # Base metadata for all plots
|
||||
├── subfolder/
|
||||
│ ├── meta.yaml # Inherits from parent, can override
|
||||
│ ├── plot1.pdf
|
||||
│ ├── plot1.yaml # Plot-specific metadata
|
||||
│ └── plot2.pdf # Uses folder metadata
|
||||
```
|
||||
|
||||
### Flexible Format Support
|
||||
- YAML files: `.yaml`, `.yml`
|
||||
- JSON files: `.json`
|
||||
- Automatic format detection based on file extension
|
||||
|
||||
### Template Integration
|
||||
- `folder_metadata`: Available in templates for folder-level metadata
|
||||
- `item.metadata`: Available for each plot in the items loop
|
||||
- Clean separation of concerns between data and presentation
|
||||
|
||||
### Performance Optimization
|
||||
- Metadata caching in `meta_cache.json` files
|
||||
- Only reload when source files are newer than cache
|
||||
- Efficient hierarchical resolution
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Folder Metadata
|
||||
```yaml
|
||||
# meta.yaml
|
||||
title: "Physics Analysis Results"
|
||||
experiment: "CMS"
|
||||
author:
|
||||
name: "Researcher Name"
|
||||
institution: "University"
|
||||
tags: ["analysis", "physics"]
|
||||
```
|
||||
|
||||
### Plot-specific Metadata
|
||||
```yaml
|
||||
# my_plot.yaml (for my_plot.pdf)
|
||||
title: "Signal Region Analysis"
|
||||
plot_type: "histogram"
|
||||
variables:
|
||||
x_axis: "mass"
|
||||
y_axis: "events"
|
||||
highlight: true
|
||||
```
|
||||
|
||||
### Template Usage
|
||||
```html
|
||||
<h1>{{ folder_metadata.title }}</h1>
|
||||
{% for item in items %}
|
||||
<div class="plot">
|
||||
<h3>{{ item.metadata.title or item.name }}</h3>
|
||||
{% if item.metadata.plot_type %}
|
||||
<span class="type">{{ item.metadata.plot_type }}</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Flexibility**: Support any metadata structure using YAML/JSON
|
||||
2. **Inheritance**: Avoid repetition by inheriting from parent folders
|
||||
3. **Override capability**: Fine-tune metadata for specific plots
|
||||
4. **Performance**: Caching system for efficient repeated builds
|
||||
5. **Validation**: Built-in error handling and validation utilities
|
||||
6. **Documentation**: Comprehensive usage documentation and examples
|
||||
|
||||
The metadata system is now fully integrated and ready for use in your scientific plot gallery generator!
|
||||
@@ -1,114 +0,0 @@
|
||||
# Metadata System Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
The metadata system allows you to add flexible metadata to your plots and folders using YAML or JSON files. Metadata is inherited hierarchically from parent folders and can be overridden at any level.
|
||||
|
||||
## File Structure
|
||||
|
||||
### Folder Metadata
|
||||
- **File names**: `meta.yaml`, `meta.yml`, or `meta.json`
|
||||
- **Location**: Place in any folder containing plots
|
||||
- **Scope**: Applies to all plots in the folder and subfolders (unless overridden)
|
||||
|
||||
### Plot-specific Metadata
|
||||
- **File names**: `{plot_name}.yaml`, `{plot_name}.yml`, or `{plot_name}.json`
|
||||
- **Location**: Place in the same folder as the plot PDF file
|
||||
- **Scope**: Applies only to the specific plot with the same name
|
||||
|
||||
## Hierarchy and Inheritance
|
||||
|
||||
1. **Root folder**: Start with folder metadata in your source directory
|
||||
2. **Subfolders**: Each subfolder can have its own `meta.yaml` that merges with parent metadata
|
||||
3. **Plot-specific**: Individual plots can have their own metadata files that override folder metadata
|
||||
|
||||
## Example Usage
|
||||
|
||||
### Folder Structure
|
||||
```
|
||||
analysis_results/
|
||||
├── meta.yaml # Root folder metadata
|
||||
├── signal/
|
||||
│ ├── meta.yaml # Signal-specific metadata
|
||||
│ ├── mass_plot.pdf
|
||||
│ └── mass_plot.yaml # Plot-specific metadata
|
||||
└── background/
|
||||
├── meta.yaml # Background-specific metadata
|
||||
└── qcd_plot.pdf
|
||||
```
|
||||
|
||||
### Example Metadata Fields
|
||||
|
||||
**Common fields for folder metadata:**
|
||||
- `title`: Folder title
|
||||
- `description`: Folder description
|
||||
- `experiment`: Experiment name (CMS, ATLAS, etc.)
|
||||
- `dataset`: Dataset identifier
|
||||
- `analysis_type`: Type of analysis
|
||||
- `author`: Author information
|
||||
- `parameters`: Analysis parameters
|
||||
- `tags`: Categorization tags
|
||||
|
||||
**Common fields for plot metadata:**
|
||||
- `plot_type`: Type of plot (histogram, scatter, etc.)
|
||||
- `variables`: Variable information (x_axis, y_axis, units)
|
||||
- `selection`: Selection criteria
|
||||
- `statistics`: Statistical information
|
||||
- `display`: Display options (highlight, featured, order_priority)
|
||||
|
||||
## Configuration
|
||||
|
||||
The metadata system can be configured in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
cache_enabled: true # Enable metadata caching
|
||||
inherit_from_parent: true # Enable hierarchical inheritance
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
### HTML Template
|
||||
Metadata is available in the HTML template as:
|
||||
- `folder_metadata`: Current folder's resolved metadata
|
||||
- `item.metadata`: Individual plot metadata (in items loop)
|
||||
|
||||
### Cache Files
|
||||
- `meta_cache.json`: Generated in each web directory
|
||||
- Contains resolved metadata for all plots in that directory
|
||||
- Used for performance optimization and debugging
|
||||
|
||||
## Usage Tips
|
||||
|
||||
1. **Start simple**: Begin with basic folder metadata and add complexity as needed
|
||||
2. **Use inheritance**: Put common metadata in parent folders to avoid repetition
|
||||
3. **Override selectively**: Use plot-specific metadata only when needed
|
||||
4. **Consistent naming**: Use consistent field names across your metadata files
|
||||
5. **Validate format**: Ensure YAML/JSON files are valid before running the generator
|
||||
|
||||
## Integration with Templates
|
||||
|
||||
In your HTML templates, you can access metadata like:
|
||||
|
||||
```html
|
||||
<!-- Folder metadata -->
|
||||
<h2>{{ folder_metadata.title }}</h2>
|
||||
<p>{{ folder_metadata.description }}</p>
|
||||
|
||||
<!-- Plot metadata -->
|
||||
{% for item in items %}
|
||||
<div class="plot-item">
|
||||
<h3>{{ item.name }}</h3>
|
||||
{% if item.metadata.plot_type %}
|
||||
<span class="plot-type">{{ item.metadata.plot_type }}</span>
|
||||
{% endif %}
|
||||
{% if item.metadata.tags %}
|
||||
<div class="tags">
|
||||
{% for tag in item.metadata.tags %}
|
||||
<span class="tag">{{ tag }}</span>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
```
|
||||
@@ -0,0 +1,136 @@
|
||||
# Gallery Sort Functionality Implementation Guide
|
||||
|
||||
## Overview
|
||||
This guide explains how to integrate the new sorting functionality that allows users to sort plots by name and creation time.
|
||||
|
||||
## Frontend Implementation (Complete ✅)
|
||||
|
||||
The frontend implementation is complete and includes:
|
||||
|
||||
### 1. Sort Controls UI
|
||||
- **Name/Time buttons**: Toggle between sorting by filename and creation time
|
||||
- **Order button**: Toggle between ascending (↑) and descending (↓) order
|
||||
- **Positioned**: Left side of the controls container, next to view toggle buttons
|
||||
- **Responsive**: Adapts to mobile layouts
|
||||
|
||||
### 2. Keyboard Shortcuts
|
||||
- `Ctrl+N`: Sort by name
|
||||
- `Ctrl+M`: Sort by time (modification/creation time)
|
||||
- `Ctrl+O`: Toggle sort order (ascending/descending)
|
||||
|
||||
### 3. Persistence
|
||||
- Sort preferences are saved to localStorage
|
||||
- Settings persist across page reloads and navigation
|
||||
|
||||
### 4. Tile Sizing
|
||||
- Grid view now shows ~6 plots per row on desktop (240px minimum width)
|
||||
- Responsive design maintains usability on mobile devices
|
||||
|
||||
## Backend Integration (Required)
|
||||
|
||||
To enable time-based sorting, you need to modify your Python gallery generation code:
|
||||
|
||||
### 1. Add Creation Time to Plot Items
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
def add_creation_time_to_items(items, base_path):
|
||||
"""Add creation time to plot items for sorting functionality."""
|
||||
for item in items:
|
||||
try:
|
||||
# Get creation time from PNG or PDF file
|
||||
png_path = None
|
||||
pdf_path = None
|
||||
|
||||
if 'png_href' in item:
|
||||
png_rel_path = item['png_href'].replace('../', '').replace('./', '')
|
||||
png_path = Path(base_path) / png_rel_path
|
||||
|
||||
if 'pdf_href' in item:
|
||||
pdf_rel_path = item['pdf_href'].replace('../', '').replace('./', '')
|
||||
pdf_path = Path(base_path) / pdf_rel_path
|
||||
|
||||
# Use PNG creation time if available, otherwise PDF
|
||||
creation_time = 0
|
||||
if png_path and png_path.exists():
|
||||
creation_time = int(png_path.stat().st_ctime)
|
||||
elif pdf_path and pdf_path.exists():
|
||||
creation_time = int(pdf_path.stat().st_ctime)
|
||||
|
||||
item['creation_time'] = creation_time
|
||||
|
||||
except Exception as e:
|
||||
print(f"Warning: Could not get creation time for {item.get('name', 'unknown')}: {e}")
|
||||
item['creation_time'] = 0
|
||||
|
||||
return items
|
||||
```
|
||||
|
||||
### 2. Integrate into Your Gallery Generation
|
||||
|
||||
In your existing gallery generation code, call this function before rendering the template:
|
||||
|
||||
```python
|
||||
# Your existing code that creates the items list
|
||||
items = generate_plot_items() # Your existing function
|
||||
|
||||
# Add creation times
|
||||
items = add_creation_time_to_items(items, gallery_base_path)
|
||||
|
||||
# Pass to template
|
||||
template.render(items=items, ...)
|
||||
```
|
||||
|
||||
### 3. Template Data Structure
|
||||
|
||||
The template now expects each item to have a `creation_time` field:
|
||||
|
||||
```python
|
||||
item = {
|
||||
'name': 'plot_name.png',
|
||||
'png_href': './plot_name.png',
|
||||
'pdf_href': './plot_name.pdf',
|
||||
'creation_time': 1642723200 # Unix timestamp
|
||||
}
|
||||
```
|
||||
|
||||
## File Locations
|
||||
|
||||
### Frontend Files (Ready to use)
|
||||
- `templates/gallery.html` - Updated with sort controls and data attributes
|
||||
- `assets/css/view-controls.css` - Styling for sort and view controls
|
||||
- `assets/css/view-override.css` - Grid layout with larger tiles
|
||||
- `assets/js/sort-manager.js` - Sort functionality implementation
|
||||
- `assets/js/gallery-app.js` - Integration of SortManager
|
||||
- `assets/js/keyboard-manager.js` - Keyboard shortcuts for sorting
|
||||
|
||||
### Backend Integration
|
||||
- `python/add_creation_time.py` - Example implementation for adding creation times
|
||||
|
||||
## Features Summary
|
||||
|
||||
### ✅ Completed Features
|
||||
1. **Larger Grid Tiles**: ~6 plots per row instead of 8
|
||||
2. **Sort Controls**: Name and time sorting with visual feedback
|
||||
3. **Sort Order Toggle**: Ascending/descending with visual indicator
|
||||
4. **Keyboard Shortcuts**: Quick access to all sort functions
|
||||
5. **Persistence**: Settings saved across sessions
|
||||
6. **Responsive Design**: Works on all screen sizes
|
||||
7. **Template Integration**: Data attributes ready for backend
|
||||
|
||||
### 🔄 Next Steps (Backend Integration)
|
||||
1. Modify your Python gallery generation code to include `creation_time`
|
||||
2. Use the provided `add_creation_time_to_items()` function
|
||||
3. Test with real plot files to ensure timestamps are correct
|
||||
|
||||
## Testing
|
||||
|
||||
After backend integration:
|
||||
1. Navigate to a gallery with multiple plots
|
||||
2. Click the sort buttons to verify functionality
|
||||
3. Use keyboard shortcuts to test responsiveness
|
||||
4. Check that sort order toggles correctly
|
||||
5. Verify settings persist after page reload
|
||||
|
||||
The frontend is fully functional and will work immediately once the backend provides the `creation_time` data.
|
||||
Reference in New Issue
Block a user