lars/ETPlot

Fork 0

Files

T

Kylian Schmidt 657b273975 [WIP] Add tests and coverage

2025-09-08 10:52:58 +02:00

6.9 KiB

Raw Blame History

Automated Coverage Testing Documentation

Overview

This repository now includes automated code coverage testing using the coverage.py package. Coverage testing helps ensure that your tests adequately exercise your codebase and identifies untested code paths.

🚀 Quick Start

Container-based Coverage (Recommended)

# Build and test with coverage in container
./tests/test_container.sh

# Or run coverage directly in container
apptainer exec gallery-generator.sif python3 /src/tests/run_coverage.py

Local Coverage Testing

# Run coverage tests locally
./tests/run_coverage_local.sh

# Or manually
pip install coverage
coverage run -m unittest tests.test_container
coverage report
coverage html

📁 Coverage Files

Core Coverage Files

.coveragerc - Coverage configuration file
tests/run_coverage.py - Automated coverage script for containers
tests/run_coverage_local.sh - Local coverage testing script

Generated Reports

coverage.xml - XML format for CI/CD integration
coverage_html_report/ - Interactive HTML reports
.coverage - Coverage data file

🔧 Configuration

Coverage Settings (`.coveragerc`)

[run]
source = .
omit = 
    tests/*          # Exclude test files
    __pycache__/*    # Exclude cache
    assets/*         # Exclude static assets
    docs/*           # Exclude documentation
    templates/*      # Exclude templates

[report]
precision = 2        # 2 decimal places
show_missing = True  # Show missing line numbers
skip_covered = False # Show all files

[html]
directory = coverage_html_report
title = Gallery Generator Coverage Report

Singularity Container Integration

The coverage package is automatically installed in the container:

pip install --no-cache-dir jinja2 pyyaml coverage

📊 Coverage Reports

Console Report

Shows coverage percentage and missing lines:

Name                    Stmts   Miss  Cover   Missing
-----------------------------------------------------
generate_gallery.py      190     45   76.32%   156-167, 234-245
orchestration/config.py    45      8   82.22%   78-82
orchestration/logger.py    67      12  82.09%   45-48, 89-94
-----------------------------------------------------
TOTAL                     302     65   78.48%

HTML Report

Interactive report with:

Line-by-line coverage highlighting
Branch coverage details
Sortable file listings
Coverage trends

XML Report

Machine-readable format for CI/CD:

GitLab CI coverage visualization
External tool integration
Coverage badges

🎯 Coverage Targets

Current Thresholds

Minimum Target: 80% overall coverage
Warning Level: Below 70% coverage
Exclusions: Test files, static assets, documentation

Best Practices

Focus on Core Logic: Prioritize business logic coverage
Test Edge Cases: Include error handling and boundary conditions
Regular Monitoring: Run coverage with every commit
Incremental Improvement: Gradually increase coverage over time

🔄 CI/CD Integration

GitLab CI Pipeline

The coverage testing is integrated into the GitLab CI pipeline:

test:coverage:
  stage: test
  script:
    - apptainer exec $CONTAINER_IMAGE python3 /src/tests/run_coverage.py
  coverage: '/TOTAL.+?(\d+\.\d+)%/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.xml

Features

Automatic Reports: Coverage reports in merge requests
Badge Integration: Coverage badges in README
Trend Tracking: Historical coverage data
Failure Thresholds: Fail builds below minimum coverage

🛠️ Advanced Usage

Custom Coverage Runs

# Test specific modules
coverage run --source=orchestration -m unittest tests.test_metadata

# Include/exclude patterns
coverage run --omit="*/tests/*" -m unittest discover

# Branch coverage (more detailed)
coverage run --branch -m unittest tests.test_container

Coverage Analysis

# Show missing lines
coverage report --show-missing

# Generate detailed HTML
coverage html --show-contexts

# Export data
coverage json
coverage xml

Integration with IDEs

VS Code: Coverage Gutters extension
PyCharm: Built-in coverage runner
Vim: Coverage highlighting plugins

📈 Coverage Metrics

What Coverage Measures

Statement Coverage: Lines of code executed
Branch Coverage: Decision paths taken
Function Coverage: Functions called
Class Coverage: Classes instantiated

What Coverage Doesn't Measure

Code Quality: Coverage ≠ good tests
Logic Correctness: 100% coverage ≠ bug-free
Performance: Execution speed not measured
Security: Vulnerabilities not detected

🧪 Testing Strategy

Container Test Suite Coverage

Current test files and their focus:

`tests/test_container.py`

Environment validation - Container setup
Utility functions - Helper functions
Metadata system - YAML processing
PDF processing - ImageMagick integration
Gallery generation - End-to-end workflow

`tests/test_build_container.py`

Container building - Singularity build process
Dependency validation - Package installation
Application functionality - Script execution

Coverage Gaps Analysis

Use tests/test_coverage.py to analyze:

Missing function coverage
Untested code paths
Critical functionality gaps
Integration test needs

🚨 Troubleshooting

Common Issues

No Coverage Data

# Ensure coverage is running tests
coverage run --debug=trace -m unittest tests.test_container

Import Errors

# Check PYTHONPATH
export PYTHONPATH=/src:$PYTHONPATH

Permission Issues

# Container write permissions
apptainer exec --writable-tmpfs container.sif python3 tests/run_coverage.py

Debug Commands

# Check coverage configuration
coverage debug config

# Verify data collection
coverage debug data

# Test discovery
coverage debug sys

📚 References

Coverage.py Documentation: https://coverage.readthedocs.io/
GitLab CI Coverage: https://docs.gitlab.com/ee/ci/testing/code_coverage.html
Testing Best Practices: Python Testing 101
Container Testing: Singularity/Apptainer Documentation

🔄 Maintenance

Regular Tasks

Weekly: Review coverage reports
Monthly: Update coverage targets
Release: Ensure minimum coverage met
Quarterly: Review exclusion patterns

Cleanup

# Remove coverage files
./tests/cleanup.sh

# Manual cleanup
rm -f .coverage coverage.xml
rm -rf coverage_html_report/

Updates

# Update coverage package
pip install --upgrade coverage

# Update container
apptainer build --force container.sif Singularity.def

This automated coverage system provides comprehensive testing insights while maintaining the containerized, dependency-free approach of the gallery generator project.

6.9 KiB Raw Blame History