-{% endfor %}
-```
-
-## Benefits
-
-1. **Flexibility**: Support any metadata structure using YAML/JSON
-2. **Inheritance**: Avoid repetition by inheriting from parent folders
-3. **Override capability**: Fine-tune metadata for specific plots
-4. **Performance**: Caching system for efficient repeated builds
-5. **Validation**: Built-in error handling and validation utilities
-6. **Documentation**: Comprehensive usage documentation and examples
-
-The metadata system is now fully integrated and ready for use in your scientific plot gallery generator!
diff --git a/docs/METADATA_USAGE.md b/docs/METADATA_USAGE.md
deleted file mode 100644
index 86a55f3..0000000
--- a/docs/METADATA_USAGE.md
+++ /dev/null
@@ -1,114 +0,0 @@
-# Metadata System Documentation
-
-## Overview
-
-The metadata system allows you to add flexible metadata to your plots and folders using YAML or JSON files. Metadata is inherited hierarchically from parent folders and can be overridden at any level.
-
-## File Structure
-
-### Folder Metadata
-- **File names**: `meta.yaml`, `meta.yml`, or `meta.json`
-- **Location**: Place in any folder containing plots
-- **Scope**: Applies to all plots in the folder and subfolders (unless overridden)
-
-### Plot-specific Metadata
-- **File names**: `{plot_name}.yaml`, `{plot_name}.yml`, or `{plot_name}.json`
-- **Location**: Place in the same folder as the plot PDF file
-- **Scope**: Applies only to the specific plot with the same name
-
-## Hierarchy and Inheritance
-
-1. **Root folder**: Start with folder metadata in your source directory
-2. **Subfolders**: Each subfolder can have its own `meta.yaml` that merges with parent metadata
-3. **Plot-specific**: Individual plots can have their own metadata files that override folder metadata
-
-## Example Usage
-
-### Folder Structure
-```
-analysis_results/
-├── meta.yaml # Root folder metadata
-├── signal/
-│ ├── meta.yaml # Signal-specific metadata
-│ ├── mass_plot.pdf
-│ └── mass_plot.yaml # Plot-specific metadata
-└── background/
- ├── meta.yaml # Background-specific metadata
- └── qcd_plot.pdf
-```
-
-### Example Metadata Fields
-
-**Common fields for folder metadata:**
-- `title`: Folder title
-- `description`: Folder description
-- `experiment`: Experiment name (CMS, ATLAS, etc.)
-- `dataset`: Dataset identifier
-- `analysis_type`: Type of analysis
-- `author`: Author information
-- `parameters`: Analysis parameters
-- `tags`: Categorization tags
-
-**Common fields for plot metadata:**
-- `plot_type`: Type of plot (histogram, scatter, etc.)
-- `variables`: Variable information (x_axis, y_axis, units)
-- `selection`: Selection criteria
-- `statistics`: Statistical information
-- `display`: Display options (highlight, featured, order_priority)
-
-## Configuration
-
-The metadata system can be configured in `config.yaml`:
-
-```yaml
-metadata:
- cache_enabled: true # Enable metadata caching
- inherit_from_parent: true # Enable hierarchical inheritance
-```
-
-## Output
-
-### HTML Template
-Metadata is available in the HTML template as:
-- `folder_metadata`: Current folder's resolved metadata
-- `item.metadata`: Individual plot metadata (in items loop)
-
-### Cache Files
-- `meta_cache.json`: Generated in each web directory
-- Contains resolved metadata for all plots in that directory
-- Used for performance optimization and debugging
-
-## Usage Tips
-
-1. **Start simple**: Begin with basic folder metadata and add complexity as needed
-2. **Use inheritance**: Put common metadata in parent folders to avoid repetition
-3. **Override selectively**: Use plot-specific metadata only when needed
-4. **Consistent naming**: Use consistent field names across your metadata files
-5. **Validate format**: Ensure YAML/JSON files are valid before running the generator
-
-## Integration with Templates
-
-In your HTML templates, you can access metadata like:
-
-```html
-
-
{{ folder_metadata.title }}
-
{{ folder_metadata.description }}
-
-
-{% for item in items %}
-
-
{{ item.name }}
- {% if item.metadata.plot_type %}
- {{ item.metadata.plot_type }}
- {% endif %}
- {% if item.metadata.tags %}
-
- {% for tag in item.metadata.tags %}
- {{ tag }}
- {% endfor %}
-
- {% endif %}
-
-{% endfor %}
-```
diff --git a/docs/SORT_IMPLEMENTATION.md b/docs/SORT_IMPLEMENTATION.md
new file mode 100644
index 0000000..b0dae4a
--- /dev/null
+++ b/docs/SORT_IMPLEMENTATION.md
@@ -0,0 +1,136 @@
+# Gallery Sort Functionality Implementation Guide
+
+## Overview
+This guide explains how to integrate the new sorting functionality that allows users to sort plots by name and creation time.
+
+## Frontend Implementation (Complete ✅)
+
+The frontend implementation is complete and includes:
+
+### 1. Sort Controls UI
+- **Name/Time buttons**: Toggle between sorting by filename and creation time
+- **Order button**: Toggle between ascending (↑) and descending (↓) order
+- **Positioned**: Left side of the controls container, next to view toggle buttons
+- **Responsive**: Adapts to mobile layouts
+
+### 2. Keyboard Shortcuts
+- `Ctrl+N`: Sort by name
+- `Ctrl+M`: Sort by time (modification/creation time)
+- `Ctrl+O`: Toggle sort order (ascending/descending)
+
+### 3. Persistence
+- Sort preferences are saved to localStorage
+- Settings persist across page reloads and navigation
+
+### 4. Tile Sizing
+- Grid view now shows ~6 plots per row on desktop (240px minimum width)
+- Responsive design maintains usability on mobile devices
+
+## Backend Integration (Required)
+
+To enable time-based sorting, you need to modify your Python gallery generation code:
+
+### 1. Add Creation Time to Plot Items
+
+```python
+from pathlib import Path
+
+def add_creation_time_to_items(items, base_path):
+ """Add creation time to plot items for sorting functionality."""
+ for item in items:
+ try:
+ # Get creation time from PNG or PDF file
+ png_path = None
+ pdf_path = None
+
+ if 'png_href' in item:
+ png_rel_path = item['png_href'].replace('../', '').replace('./', '')
+ png_path = Path(base_path) / png_rel_path
+
+ if 'pdf_href' in item:
+ pdf_rel_path = item['pdf_href'].replace('../', '').replace('./', '')
+ pdf_path = Path(base_path) / pdf_rel_path
+
+ # Use PNG creation time if available, otherwise PDF
+ creation_time = 0
+ if png_path and png_path.exists():
+ creation_time = int(png_path.stat().st_ctime)
+ elif pdf_path and pdf_path.exists():
+ creation_time = int(pdf_path.stat().st_ctime)
+
+ item['creation_time'] = creation_time
+
+ except Exception as e:
+ print(f"Warning: Could not get creation time for {item.get('name', 'unknown')}: {e}")
+ item['creation_time'] = 0
+
+ return items
+```
+
+### 2. Integrate into Your Gallery Generation
+
+In your existing gallery generation code, call this function before rendering the template:
+
+```python
+# Your existing code that creates the items list
+items = generate_plot_items() # Your existing function
+
+# Add creation times
+items = add_creation_time_to_items(items, gallery_base_path)
+
+# Pass to template
+template.render(items=items, ...)
+```
+
+### 3. Template Data Structure
+
+The template now expects each item to have a `creation_time` field:
+
+```python
+item = {
+ 'name': 'plot_name.png',
+ 'png_href': './plot_name.png',
+ 'pdf_href': './plot_name.pdf',
+ 'creation_time': 1642723200 # Unix timestamp
+}
+```
+
+## File Locations
+
+### Frontend Files (Ready to use)
+- `templates/gallery.html` - Updated with sort controls and data attributes
+- `assets/css/view-controls.css` - Styling for sort and view controls
+- `assets/css/view-override.css` - Grid layout with larger tiles
+- `assets/js/sort-manager.js` - Sort functionality implementation
+- `assets/js/gallery-app.js` - Integration of SortManager
+- `assets/js/keyboard-manager.js` - Keyboard shortcuts for sorting
+
+### Backend Integration
+- `python/add_creation_time.py` - Example implementation for adding creation times
+
+## Features Summary
+
+### ✅ Completed Features
+1. **Larger Grid Tiles**: ~6 plots per row instead of 8
+2. **Sort Controls**: Name and time sorting with visual feedback
+3. **Sort Order Toggle**: Ascending/descending with visual indicator
+4. **Keyboard Shortcuts**: Quick access to all sort functions
+5. **Persistence**: Settings saved across sessions
+6. **Responsive Design**: Works on all screen sizes
+7. **Template Integration**: Data attributes ready for backend
+
+### 🔄 Next Steps (Backend Integration)
+1. Modify your Python gallery generation code to include `creation_time`
+2. Use the provided `add_creation_time_to_items()` function
+3. Test with real plot files to ensure timestamps are correct
+
+## Testing
+
+After backend integration:
+1. Navigate to a gallery with multiple plots
+2. Click the sort buttons to verify functionality
+3. Use keyboard shortcuts to test responsiveness
+4. Check that sort order toggles correctly
+5. Verify settings persist after page reload
+
+The frontend is fully functional and will work immediately once the backend provides the `creation_time` data.
diff --git a/examples/metadata.json b/examples/metadata.json
new file mode 100644
index 0000000..61751f9
--- /dev/null
+++ b/examples/metadata.json
@@ -0,0 +1,19 @@
+{
+ "title": "Sample Research Project",
+ "description": "This folder contains plots and analysis results for our sample research project studying data visualization techniques.",
+ "author": "Research Team",
+ "created": "2025-01-15",
+ "project_id": "PROJ-2025-001",
+ "status": "Active",
+ "tags": ["data-science", "visualization", "research"],
+ "collaborators": ["Alice Smith", "Bob Johnson", "Carol Davis"],
+ "funding": {
+ "agency": "National Science Foundation",
+ "grant_number": "NSF-2025-12345",
+ "amount": "$150,000"
+ },
+ "contact": "research-team@example.com",
+ "repository": "https://github.com/example/sample-project",
+ "documentation": "https://docs.example.com/sample-project",
+ "notes": "This is a long note that demonstrates how the system handles lengthy text content. It should be truncated in the display and allow users to expand it to see the full content. This helps keep the interface clean while still providing access to detailed information when needed."
+}
diff --git a/export_api.py b/export_api.py
deleted file mode 100644
index c2385f7..0000000
--- a/export_api.py
+++ /dev/null
@@ -1,94 +0,0 @@
-"""
-Simple Flask API for PDF Export
-
-This provides a web API endpoint for the gallery export functionality.
-"""
-
-from flask import Flask, request, jsonify, send_file
-from pathlib import Path
-import json
-import tempfile
-import os
-from orchestration.pdf_export import PDFExporter, check_dependencies
-
-app = Flask(__name__)
-exporter = PDFExporter()
-
-
-@app.route('/api/export-pdf', methods=['POST'])
-def export_pdf():
- """Export selected plots to merged PDF"""
- try:
- data = request.get_json()
-
- if not data or 'plots' not in data:
- return jsonify({'error': 'No plots specified'}), 400
-
- plot_urls = data['plots']
- layout = data.get('layout', {'rows': 2, 'cols': 2})
-
- # Convert URLs to file paths
- plot_paths = []
- for url in plot_urls:
- # Extract path from file:// URL or relative path
- if url.startswith('file://'):
- path = url[7:] # Remove 'file://' prefix
- elif url.startswith('http'):
- return jsonify({'error': 'Remote URLs not supported'}), 400
- else:
- # Assume relative path from web root
- path = url
-
- # Resolve to absolute path
- plot_path = Path(path)
- if not plot_path.exists():
- return jsonify({'error': f'Plot not found: {path}'}), 404
-
- plot_paths.append(str(plot_path))
-
- if not plot_paths:
- return jsonify({'error': 'No valid plots found'}), 400
-
- # Create merged PDF
- pdf_bytes = exporter.merge_plots(plot_paths, layout)
-
- # Create temporary file to serve
- with tempfile.NamedTemporaryFile(suffix='.pdf', delete=False) as tmp_file:
- tmp_file.write(pdf_bytes)
- tmp_path = tmp_file.name
-
- def cleanup_temp_file():
- """Clean up temporary file after sending"""
- try:
- os.unlink(tmp_path)
- except OSError:
- pass
-
- return send_file(
- tmp_path,
- as_attachment=True,
- download_name='merged_plots.pdf',
- mimetype='application/pdf'
- )
-
- except Exception as e:
- return jsonify({'error': str(e)}), 500
-
-
-@app.route('/api/export-status', methods=['GET'])
-def export_status():
- """Check export capabilities"""
- deps = check_dependencies()
- return jsonify({
- 'available': any(deps.values()),
- 'dependencies': deps,
- 'methods': {
- 'pypdf2': deps['pypdf2'],
- 'pdfjam': deps['pdfjam']
- }
- })
-
-
-if __name__ == '__main__':
- # For development only
- app.run(debug=True, port=5000)
diff --git a/export_plots.py b/export_plots.py
deleted file mode 100644
index 4c4fb0a..0000000
--- a/export_plots.py
+++ /dev/null
@@ -1,107 +0,0 @@
-#!/usr/bin/env python3
-"""
-Standalone PDF Export Script
-
-This script processes export requests from the gallery and creates merged PDFs.
-Usage: python export_plots.py [request_file.json]
-"""
-
-import sys
-import json
-import argparse
-from pathlib import Path
-from orchestration.pdf_export import PDFExporter, check_dependencies
-
-
-def main():
- parser = argparse.ArgumentParser(description='Export gallery plots to merged PDF')
- parser.add_argument('request_file', nargs='?', default='export_request.json',
- help='JSON file containing export request')
- parser.add_argument('--output', '-o', help='Output PDF file path')
- parser.add_argument('--check-deps', action='store_true',
- help='Check available dependencies')
-
- args = parser.parse_args()
-
- if args.check_deps:
- deps = check_dependencies()
- print("Available dependencies:")
- for dep, available in deps.items():
- status = "✓" if available else "✗"
- print(f" {status} {dep}")
-
- if not any(deps.values()):
- print("\nNo PDF merging tools available!")
- print("Install one of the following:")
- print(" - PyPDF2 and reportlab: pip install PyPDF2 reportlab")
- print(" - pdfjam: apt-get install texlive-extra-utils (on Ubuntu/Debian)")
- return
-
- request_file = Path(args.request_file)
- if not request_file.exists():
- print(f"Error: Request file '{request_file}' not found")
- print("Create a JSON file with the following structure:")
- print(json.dumps({
- "plots": ["/path/to/plot1.pdf", "/path/to/plot2.pdf"],
- "layout": {"rows": 1, "cols": 2},
- "output_name": "merged_plots.pdf"
- }, indent=2))
- return 1
-
- try:
- with open(request_file, 'r') as f:
- request_data = json.load(f)
- except json.JSONDecodeError as e:
- print(f"Error: Invalid JSON in '{request_file}': {e}")
- return 1
-
- # Validate request data
- if 'plots' not in request_data:
- print("Error: Missing 'plots' field in request")
- return 1
-
- plot_paths = request_data['plots']
- layout = request_data.get('layout', {'rows': 2, 'cols': 2})
- output_name = args.output or request_data.get('output_name', 'merged_plots.pdf')
-
- # Validate plot files exist
- missing_files = []
- for plot_path in plot_paths:
- if not Path(plot_path).exists():
- missing_files.append(plot_path)
-
- if missing_files:
- print("Error: The following plot files were not found:")
- for missing in missing_files:
- print(f" - {missing}")
- return 1
-
- print(f"Exporting {len(plot_paths)} plots to '{output_name}'...")
- print(f"Layout: {layout['rows']}x{layout['cols']}")
-
- # Create exporter and merge plots
- exporter = PDFExporter()
-
- try:
- pdf_bytes = exporter.merge_plots(plot_paths, layout, output_name)
-
- if not args.output and 'output_name' not in request_data:
- # Write to file if not already done by exporter
- with open(output_name, 'wb') as f:
- f.write(pdf_bytes)
-
- print(f"✓ Successfully created '{output_name}'")
- print(f" File size: {len(pdf_bytes) / 1024:.1f} KB")
-
- # Clean up request file if export was successful
- request_file.unlink(missing_ok=True)
-
- except Exception as e:
- print(f"Error during export: {e}")
- return 1
-
- return 0
-
-
-if __name__ == '__main__':
- sys.exit(main())
diff --git a/generate_gallery.py b/generate_gallery.py
index 65b622c..3eae8b5 100644
--- a/generate_gallery.py
+++ b/generate_gallery.py
@@ -19,6 +19,7 @@ import os
import sys
from pathlib import Path
from typing import Dict, Any, Optional
+from datetime import datetime
from jinja2 import Environment, FileSystemLoader
from orchestration.config import Config
@@ -32,7 +33,20 @@ from orchestration.metadata import (
CONFIG = Config.from_yaml("config.yaml")
+
+def datetime_from_timestamp(timestamp):
+ """Convert a Unix timestamp to a datetime object."""
+ return datetime.fromtimestamp(timestamp)
+
+
+def strftime_filter(dt, fmt):
+ """Format a datetime object using strftime."""
+ return dt.strftime(fmt)
+
+
env = Environment(loader=FileSystemLoader("."))
+env.filters['datetime_from_timestamp'] = datetime_from_timestamp
+env.filters['strftime'] = strftime_filter
template = env.get_template("templates/gallery.html")
@@ -147,11 +161,15 @@ def build_gallery(source_dir: Path, web_dir: Path,
plot_metadata = resolve_metadata_for_plot(pdf_file, current_metadata)
plot_metadata_cache[pdf_file.stem] = plot_metadata
+ # Get source file creation time (in seconds since epoch)
+ source_creation_time = int(pdf_file.stat().st_ctime)
+
items.append({
"name": pdf_file.stem,
"pdf_href": pdf_file.name,
"png_href": png_file.name,
- "metadata": plot_metadata
+ "metadata": plot_metadata,
+ "creation_time": source_creation_time
})
# Save metadata cache for this directory
@@ -357,10 +375,14 @@ def main(clean_first: bool = False) -> None:
else:
print(f"Skipping {source_png_path.name} (up to date)")
+ # Get source file creation time for single file
+ source_creation_time = int(source_path.stat().st_ctime)
+
items = [{
"name": source_path.stem,
"pdf_href": pdf_name,
- "png_href": png_name
+ "png_href": png_name,
+ "creation_time": source_creation_time
}]
# Calculate statistics for single file
diff --git a/orchestration/metadata.py b/orchestration/metadata.py
index 4c1d9b8..a6bfb84 100644
--- a/orchestration/metadata.py
+++ b/orchestration/metadata.py
@@ -50,7 +50,7 @@ def load_metadata_file(metadata_path: Path) -> Dict[str, Any]:
def load_folder_metadata(folder_path: Path) -> Dict[str, Any]:
"""
- Load folder-level metadata from meta.yaml or meta.json.
+ Load folder-level metadata from meta.yaml, meta.json, or metadata.json.
Args:
folder_path: Path to the folder to check for metadata
@@ -58,8 +58,8 @@ def load_folder_metadata(folder_path: Path) -> Dict[str, Any]:
Returns:
Dictionary containing the folder metadata
"""
- # Try YAML first, then JSON
- for filename in ['meta.yaml', 'meta.yml', 'meta.json']:
+ # Try YAML first, then JSON (including metadata.json for backwards compatibility)
+ for filename in ['meta.yaml', 'meta.yml', 'meta.json', 'metadata.json']:
metadata_path = folder_path / filename
if metadata_path.exists():
return load_metadata_file(metadata_path)
diff --git a/python/add_creation_time.py b/python/add_creation_time.py
new file mode 100644
index 0000000..135d807
--- /dev/null
+++ b/python/add_creation_time.py
@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+"""
+Gallery Creation Time Integration
+Example function to add creation time metadata to plot items.
+This should be integrated into your existing Python gallery generation code.
+"""
+
+from pathlib import Path
+
+
+def add_creation_time_to_items(items, base_path):
+ """
+ Add creation time to plot items for sorting functionality.
+
+ Args:
+ items: List of plot item dictionaries
+ base_path: Base path where plot files are located
+
+ Returns:
+ Updated items list with creation_time field
+ """
+ for item in items:
+ try:
+ # Try to get creation time from PNG file first, then PDF
+ png_path = None
+ pdf_path = None
+
+ # Extract relative path from href
+ if 'png_href' in item:
+ png_rel_path = item['png_href'].replace('../', '').replace(
+ './', '')
+ png_path = Path(base_path) / png_rel_path
+
+ if 'pdf_href' in item:
+ pdf_rel_path = item['pdf_href'].replace('../', '').replace(
+ './', '')
+ pdf_path = Path(base_path) / pdf_rel_path
+
+ # Use PNG creation time if available, otherwise PDF
+ creation_time = 0
+ if png_path and png_path.exists():
+ creation_time = int(png_path.stat().st_ctime)
+ elif pdf_path and pdf_path.exists():
+ creation_time = int(pdf_path.stat().st_ctime)
+
+ # Add creation time as timestamp (JavaScript can handle this)
+ item['creation_time'] = creation_time
+
+ except Exception as e:
+ # Fallback to 0 if there's any error
+ name = item.get('name', 'unknown')
+ print(f"Warning: Could not get creation time for {name}: {e}")
+ item['creation_time'] = 0
+
+ return items
+
+
+def example_integration():
+ """
+ Example of how to integrate this into your existing gallery generation.
+ """
+ # This would be part of your existing gallery generation code
+ items = [
+ {
+ 'name': 'plot1.png',
+ 'png_href': './plot1.png',
+ 'pdf_href': './plot1.pdf'
+ },
+ {
+ 'name': 'plot2.png',
+ 'png_href': './plot2.png',
+ 'pdf_href': './plot2.pdf'
+ }
+ ]
+
+ base_path = "/path/to/your/gallery/directory"
+
+ # Add creation times
+ items_with_time = add_creation_time_to_items(items, base_path)
+
+ # Now items_with_time can be passed to your Jinja2 template
+ # The template will have access to item.creation_time for each item
+
+ return items_with_time
+
+
+if __name__ == "__main__":
+ # Test the function
+ items = example_integration()
+ for item in items:
+ created = item['creation_time']
+ print(f"Plot: {item['name']}, Created: {created}")
diff --git a/python/add_metadata.py b/python/add_metadata.py
new file mode 100644
index 0000000..cb2d5ef
--- /dev/null
+++ b/python/add_metadata.py
@@ -0,0 +1,45 @@
+from typing import Dict, Any
+import json
+from pathlib import Path
+
+
+def open_metadata(path: str, filename: str = "metadata.json") -> Dict[str, Any]:
+ """
+ Open metadata file and return its contents as a dictionary.
+
+ Args:
+ path: The directory path where the metadata file is located.
+ filename: The name of the metadata file (default: "metadata.json").
+
+ Returns:
+ A dictionary containing the metadata.
+ """
+ with open(Path(path) / filename, 'r', encoding='utf-8') as f:
+ try:
+ return json.load(f)
+ except json.JSONDecodeError as e:
+ raise ValueError(f"Error decoding JSON from {filename}: {e}")
+ except FileNotFoundError:
+ raise FileNotFoundError(f"Metadata file {filename} not found in {path}")
+ except Exception as e:
+ raise RuntimeError(f"Unexpected error reading metadata: {e}")
+
+
+def merge_metadata(
+ base_metadata: Dict[str, Any],
+ additional_metadata: Dict[str, Any]
+) -> Dict[str, Any]:
+ """
+ Merge two metadata dictionaries.
+
+ Args:
+ base_metadata: The base metadata dictionary.
+ additional_metadata: The additional metadata dictionary to merge.
+
+ Returns:
+ A new dictionary containing the merged metadata.
+ """
+ merged = base_metadata.copy()
+ for key, value in additional_metadata.items():
+ merged[key] = value
+ return merged
diff --git a/templates/gallery.html b/templates/gallery.html
index 927df73..a7d3d07 100644
--- a/templates/gallery.html
+++ b/templates/gallery.html
@@ -34,50 +34,118 @@
+
+ {% if folder_metadata %}
+
+
+
+
+ {% for key, value in folder_metadata.items() %}
+
+ {{ key }}:
+
+ {% if value is string and (value.startswith('http://') or value.startswith('https://')) %}
+ {{ value }}
+ {% elif value is string and value|length > 100 %}
+ {{ value[:100] }}...
+
+ {{ value }}
+ {% elif value is iterable and value is not string and value is not mapping %}
+
+ {% for item in value %}
+ {{ item }}
+ {% endfor %}
+
{% endfor %}
@@ -118,9 +186,6 @@
-