Remove pre-package-restructure cruft and dead features
Delete the top-level implementation superseded by the gallery/ package conversion (generate_gallery.py, orchestration/, root templates/ and assets/, python/ scripts), stray scratch files, and docs describing a container/GitLab-CI coverage workflow that no longer exists. Also drop two half-wired, never-invoked features: the backup_folder config/TUI option (create_backup() was never called from the pipeline) and the unfinished export-to-LaTeX JS/CSS. Update README's install instructions to the current Gitea remote.
This commit is contained in:
@@ -1,275 +0,0 @@
|
||||
# Automated Coverage Testing Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
This repository now includes automated code coverage testing using the `coverage.py` package. Coverage testing helps ensure that your tests adequately exercise your codebase and identifies untested code paths.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Container-based Coverage (Recommended)
|
||||
```bash
|
||||
# Build and test with coverage in container
|
||||
./tests/test_container.sh
|
||||
|
||||
# Or run coverage directly in container
|
||||
apptainer exec gallery-generator.sif python3 /src/tests/run_coverage.py
|
||||
```
|
||||
|
||||
### Local Coverage Testing
|
||||
```bash
|
||||
# Run coverage tests locally
|
||||
./tests/run_coverage_local.sh
|
||||
|
||||
# Or manually
|
||||
pip install coverage
|
||||
coverage run -m unittest tests.test_container
|
||||
coverage report
|
||||
coverage html
|
||||
```
|
||||
|
||||
## 📁 Coverage Files
|
||||
|
||||
### Core Coverage Files
|
||||
- **`.coveragerc`** - Coverage configuration file
|
||||
- **`tests/run_coverage.py`** - Automated coverage script for containers
|
||||
- **`tests/run_coverage_local.sh`** - Local coverage testing script
|
||||
|
||||
### Generated Reports
|
||||
- **`coverage.xml`** - XML format for CI/CD integration
|
||||
- **`coverage_html_report/`** - Interactive HTML reports
|
||||
- **`.coverage`** - Coverage data file
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### Coverage Settings (`.coveragerc`)
|
||||
```ini
|
||||
[run]
|
||||
source = .
|
||||
omit =
|
||||
tests/* # Exclude test files
|
||||
__pycache__/* # Exclude cache
|
||||
assets/* # Exclude static assets
|
||||
docs/* # Exclude documentation
|
||||
templates/* # Exclude templates
|
||||
|
||||
[report]
|
||||
precision = 2 # 2 decimal places
|
||||
show_missing = True # Show missing line numbers
|
||||
skip_covered = False # Show all files
|
||||
|
||||
[html]
|
||||
directory = coverage_html_report
|
||||
title = Gallery Generator Coverage Report
|
||||
```
|
||||
|
||||
### Singularity Container Integration
|
||||
The coverage package is automatically installed in the container:
|
||||
```bash
|
||||
pip install --no-cache-dir jinja2 pyyaml coverage
|
||||
```
|
||||
|
||||
## 📊 Coverage Reports
|
||||
|
||||
### Console Report
|
||||
Shows coverage percentage and missing lines:
|
||||
```
|
||||
Name Stmts Miss Cover Missing
|
||||
-----------------------------------------------------
|
||||
generate_gallery.py 190 45 76.32% 156-167, 234-245
|
||||
orchestration/config.py 45 8 82.22% 78-82
|
||||
orchestration/logger.py 67 12 82.09% 45-48, 89-94
|
||||
-----------------------------------------------------
|
||||
TOTAL 302 65 78.48%
|
||||
```
|
||||
|
||||
### HTML Report
|
||||
Interactive report with:
|
||||
- Line-by-line coverage highlighting
|
||||
- Branch coverage details
|
||||
- Sortable file listings
|
||||
- Coverage trends
|
||||
|
||||
### XML Report
|
||||
Machine-readable format for CI/CD:
|
||||
- GitLab CI coverage visualization
|
||||
- External tool integration
|
||||
- Coverage badges
|
||||
|
||||
## 🎯 Coverage Targets
|
||||
|
||||
### Current Thresholds
|
||||
- **Minimum Target**: 80% overall coverage
|
||||
- **Warning Level**: Below 70% coverage
|
||||
- **Exclusions**: Test files, static assets, documentation
|
||||
|
||||
### Best Practices
|
||||
- **Focus on Core Logic**: Prioritize business logic coverage
|
||||
- **Test Edge Cases**: Include error handling and boundary conditions
|
||||
- **Regular Monitoring**: Run coverage with every commit
|
||||
- **Incremental Improvement**: Gradually increase coverage over time
|
||||
|
||||
## 🔄 CI/CD Integration
|
||||
|
||||
### GitLab CI Pipeline
|
||||
The coverage testing is integrated into the GitLab CI pipeline:
|
||||
|
||||
```yaml
|
||||
test:coverage:
|
||||
stage: test
|
||||
script:
|
||||
- apptainer exec $CONTAINER_IMAGE python3 /src/tests/run_coverage.py
|
||||
coverage: '/TOTAL.+?(\d+\.\d+)%/'
|
||||
artifacts:
|
||||
reports:
|
||||
coverage_report:
|
||||
coverage_format: cobertura
|
||||
path: coverage.xml
|
||||
```
|
||||
|
||||
### Features
|
||||
- **Automatic Reports**: Coverage reports in merge requests
|
||||
- **Badge Integration**: Coverage badges in README
|
||||
- **Trend Tracking**: Historical coverage data
|
||||
- **Failure Thresholds**: Fail builds below minimum coverage
|
||||
|
||||
## 🛠️ Advanced Usage
|
||||
|
||||
### Custom Coverage Runs
|
||||
```bash
|
||||
# Test specific modules
|
||||
coverage run --source=orchestration -m unittest tests.test_metadata
|
||||
|
||||
# Include/exclude patterns
|
||||
coverage run --omit="*/tests/*" -m unittest discover
|
||||
|
||||
# Branch coverage (more detailed)
|
||||
coverage run --branch -m unittest tests.test_container
|
||||
```
|
||||
|
||||
### Coverage Analysis
|
||||
```bash
|
||||
# Show missing lines
|
||||
coverage report --show-missing
|
||||
|
||||
# Generate detailed HTML
|
||||
coverage html --show-contexts
|
||||
|
||||
# Export data
|
||||
coverage json
|
||||
coverage xml
|
||||
```
|
||||
|
||||
### Integration with IDEs
|
||||
- **VS Code**: Coverage Gutters extension
|
||||
- **PyCharm**: Built-in coverage runner
|
||||
- **Vim**: Coverage highlighting plugins
|
||||
|
||||
## 📈 Coverage Metrics
|
||||
|
||||
### What Coverage Measures
|
||||
- **Statement Coverage**: Lines of code executed
|
||||
- **Branch Coverage**: Decision paths taken
|
||||
- **Function Coverage**: Functions called
|
||||
- **Class Coverage**: Classes instantiated
|
||||
|
||||
### What Coverage Doesn't Measure
|
||||
- **Code Quality**: Coverage ≠ good tests
|
||||
- **Logic Correctness**: 100% coverage ≠ bug-free
|
||||
- **Performance**: Execution speed not measured
|
||||
- **Security**: Vulnerabilities not detected
|
||||
|
||||
## 🧪 Testing Strategy
|
||||
|
||||
### Container Test Suite Coverage
|
||||
Current test files and their focus:
|
||||
|
||||
#### `tests/test_container.py`
|
||||
- **Environment validation** - Container setup
|
||||
- **Utility functions** - Helper functions
|
||||
- **Metadata system** - YAML processing
|
||||
- **PDF processing** - ImageMagick integration
|
||||
- **Gallery generation** - End-to-end workflow
|
||||
|
||||
#### `tests/test_build_container.py`
|
||||
- **Container building** - Singularity build process
|
||||
- **Dependency validation** - Package installation
|
||||
- **Application functionality** - Script execution
|
||||
|
||||
### Coverage Gaps Analysis
|
||||
Use `tests/test_coverage.py` to analyze:
|
||||
- Missing function coverage
|
||||
- Untested code paths
|
||||
- Critical functionality gaps
|
||||
- Integration test needs
|
||||
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### No Coverage Data
|
||||
```bash
|
||||
# Ensure coverage is running tests
|
||||
coverage run --debug=trace -m unittest tests.test_container
|
||||
```
|
||||
|
||||
#### Import Errors
|
||||
```bash
|
||||
# Check PYTHONPATH
|
||||
export PYTHONPATH=/src:$PYTHONPATH
|
||||
```
|
||||
|
||||
#### Permission Issues
|
||||
```bash
|
||||
# Container write permissions
|
||||
apptainer exec --writable-tmpfs container.sif python3 tests/run_coverage.py
|
||||
```
|
||||
|
||||
### Debug Commands
|
||||
```bash
|
||||
# Check coverage configuration
|
||||
coverage debug config
|
||||
|
||||
# Verify data collection
|
||||
coverage debug data
|
||||
|
||||
# Test discovery
|
||||
coverage debug sys
|
||||
```
|
||||
|
||||
## 📚 References
|
||||
|
||||
- **Coverage.py Documentation**: https://coverage.readthedocs.io/
|
||||
- **GitLab CI Coverage**: https://docs.gitlab.com/ee/ci/testing/code_coverage.html
|
||||
- **Testing Best Practices**: Python Testing 101
|
||||
- **Container Testing**: Singularity/Apptainer Documentation
|
||||
|
||||
## 🔄 Maintenance
|
||||
|
||||
### Regular Tasks
|
||||
- **Weekly**: Review coverage reports
|
||||
- **Monthly**: Update coverage targets
|
||||
- **Release**: Ensure minimum coverage met
|
||||
- **Quarterly**: Review exclusion patterns
|
||||
|
||||
### Cleanup
|
||||
```bash
|
||||
# Remove coverage files
|
||||
./tests/cleanup.sh
|
||||
|
||||
# Manual cleanup
|
||||
rm -f .coverage coverage.xml
|
||||
rm -rf coverage_html_report/
|
||||
```
|
||||
|
||||
### Updates
|
||||
```bash
|
||||
# Update coverage package
|
||||
pip install --upgrade coverage
|
||||
|
||||
# Update container
|
||||
apptainer build --force container.sif Singularity.def
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*This automated coverage system provides comprehensive testing insights while maintaining the containerized, dependency-free approach of the gallery generator project.*
|
||||
@@ -1,136 +0,0 @@
|
||||
# Gallery Sort Functionality Implementation Guide
|
||||
|
||||
## Overview
|
||||
This guide explains how to integrate the new sorting functionality that allows users to sort plots by name and creation time.
|
||||
|
||||
## Frontend Implementation (Complete ✅)
|
||||
|
||||
The frontend implementation is complete and includes:
|
||||
|
||||
### 1. Sort Controls UI
|
||||
- **Name/Time buttons**: Toggle between sorting by filename and creation time
|
||||
- **Order button**: Toggle between ascending (↑) and descending (↓) order
|
||||
- **Positioned**: Left side of the controls container, next to view toggle buttons
|
||||
- **Responsive**: Adapts to mobile layouts
|
||||
|
||||
### 2. Keyboard Shortcuts
|
||||
- `Ctrl+N`: Sort by name
|
||||
- `Ctrl+M`: Sort by time (modification/creation time)
|
||||
- `Ctrl+O`: Toggle sort order (ascending/descending)
|
||||
|
||||
### 3. Persistence
|
||||
- Sort preferences are saved to localStorage
|
||||
- Settings persist across page reloads and navigation
|
||||
|
||||
### 4. Tile Sizing
|
||||
- Grid view now shows ~6 plots per row on desktop (240px minimum width)
|
||||
- Responsive design maintains usability on mobile devices
|
||||
|
||||
## Backend Integration (Required)
|
||||
|
||||
To enable time-based sorting, you need to modify your Python gallery generation code:
|
||||
|
||||
### 1. Add Creation Time to Plot Items
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
def add_creation_time_to_items(items, base_path):
|
||||
"""Add creation time to plot items for sorting functionality."""
|
||||
for item in items:
|
||||
try:
|
||||
# Get creation time from PNG or PDF file
|
||||
png_path = None
|
||||
pdf_path = None
|
||||
|
||||
if 'png_href' in item:
|
||||
png_rel_path = item['png_href'].replace('../', '').replace('./', '')
|
||||
png_path = Path(base_path) / png_rel_path
|
||||
|
||||
if 'pdf_href' in item:
|
||||
pdf_rel_path = item['pdf_href'].replace('../', '').replace('./', '')
|
||||
pdf_path = Path(base_path) / pdf_rel_path
|
||||
|
||||
# Use PNG creation time if available, otherwise PDF
|
||||
creation_time = 0
|
||||
if png_path and png_path.exists():
|
||||
creation_time = int(png_path.stat().st_ctime)
|
||||
elif pdf_path and pdf_path.exists():
|
||||
creation_time = int(pdf_path.stat().st_ctime)
|
||||
|
||||
item['creation_time'] = creation_time
|
||||
|
||||
except Exception as e:
|
||||
print(f"Warning: Could not get creation time for {item.get('name', 'unknown')}: {e}")
|
||||
item['creation_time'] = 0
|
||||
|
||||
return items
|
||||
```
|
||||
|
||||
### 2. Integrate into Your Gallery Generation
|
||||
|
||||
In your existing gallery generation code, call this function before rendering the template:
|
||||
|
||||
```python
|
||||
# Your existing code that creates the items list
|
||||
items = generate_plot_items() # Your existing function
|
||||
|
||||
# Add creation times
|
||||
items = add_creation_time_to_items(items, gallery_base_path)
|
||||
|
||||
# Pass to template
|
||||
template.render(items=items, ...)
|
||||
```
|
||||
|
||||
### 3. Template Data Structure
|
||||
|
||||
The template now expects each item to have a `creation_time` field:
|
||||
|
||||
```python
|
||||
item = {
|
||||
'name': 'plot_name.png',
|
||||
'png_href': './plot_name.png',
|
||||
'pdf_href': './plot_name.pdf',
|
||||
'creation_time': 1642723200 # Unix timestamp
|
||||
}
|
||||
```
|
||||
|
||||
## File Locations
|
||||
|
||||
### Frontend Files (Ready to use)
|
||||
- `templates/gallery.html` - Updated with sort controls and data attributes
|
||||
- `assets/css/view-controls.css` - Styling for sort and view controls
|
||||
- `assets/css/view-override.css` - Grid layout with larger tiles
|
||||
- `assets/js/sort-manager.js` - Sort functionality implementation
|
||||
- `assets/js/gallery-app.js` - Integration of SortManager
|
||||
- `assets/js/keyboard-manager.js` - Keyboard shortcuts for sorting
|
||||
|
||||
### Backend Integration
|
||||
- `python/add_creation_time.py` - Example implementation for adding creation times
|
||||
|
||||
## Features Summary
|
||||
|
||||
### ✅ Completed Features
|
||||
1. **Larger Grid Tiles**: ~6 plots per row instead of 8
|
||||
2. **Sort Controls**: Name and time sorting with visual feedback
|
||||
3. **Sort Order Toggle**: Ascending/descending with visual indicator
|
||||
4. **Keyboard Shortcuts**: Quick access to all sort functions
|
||||
5. **Persistence**: Settings saved across sessions
|
||||
6. **Responsive Design**: Works on all screen sizes
|
||||
7. **Template Integration**: Data attributes ready for backend
|
||||
|
||||
### 🔄 Next Steps (Backend Integration)
|
||||
1. Modify your Python gallery generation code to include `creation_time`
|
||||
2. Use the provided `add_creation_time_to_items()` function
|
||||
3. Test with real plot files to ensure timestamps are correct
|
||||
|
||||
## Testing
|
||||
|
||||
After backend integration:
|
||||
1. Navigate to a gallery with multiple plots
|
||||
2. Click the sort buttons to verify functionality
|
||||
3. Use keyboard shortcuts to test responsiveness
|
||||
4. Check that sort order toggles correctly
|
||||
5. Verify settings persist after page reload
|
||||
|
||||
The frontend is fully functional and will work immediately once the backend provides the `creation_time` data.
|
||||
Reference in New Issue
Block a user