Code Coverage Checker Laravel Package

Technical Evaluation

Architecture Fit

• Pros:

  • Composable with existing CI/CD pipelines: Integrates seamlessly with PHPUnit, a core dependency in Laravel projects, making it a natural fit for enforcing coverage thresholds in test suites.
  • Granularity: Supports line, method, and class coverage metrics, allowing alignment with team-specific quality standards (e.g., enforcing 90% line coverage for critical modules).
  • Symfony/PHPUnit Bridge: Explicit support for Symfony’s PHPUnit bridge ensures compatibility with Laravel’s testing ecosystem, which often relies on Symfony components (e.g., symfony/phpunit-bridge for kernel testing).
  • Directory Filtering: Ability to enforce thresholds on specific directories (e.g., app/, src/) enables targeted quality gates for high-risk or business-critical code.

• Cons:

  • No native Laravel integration: Requires manual CLI invocation (e.g., vendor/bin/code-coverage-checker), which may not align with Laravel’s service container or task scheduling (e.g., Artisan commands).
  • Limited visualization: Output is CLI-based; lacks integration with Laravel’s existing dashboards (e.g., Forge, Nova, or third-party tools like SonarQube).
  • No dynamic thresholding: Thresholds are static (hardcoded in CLI calls), which may not suit projects needing context-aware coverage (e.g., higher thresholds for new features).

Integration Feasibility

• PHPUnit Dependency: Since Laravel projects already use PHPUnit (via laravel/framework), this package adds minimal overhead.
• Coverage Report Generation: Requires PHPUnit’s --coverage-php flag, which is standard in Laravel’s phpunit.xml configurations.
• CI/CD Hooks: Can be easily added to GitHub Actions, GitLab CI, or Laravel Forge deploy scripts as a pre-commit or pre-deploy gate.
• Example Integration:

  # GitHub Actions example
  jobs:
    test:
      runs-on: ubuntu-latest
      steps:
        - run: phpunit --coverage-php=coverage.clover
        - run: vendor/bin/code-coverage-checker coverage.clover line 90
  

Technical Risk

• Version Compatibility:
  • Supports PHP 8+, PHPUnit 9+, and Symfony 6–8, which aligns with Laravel’s current LTS (Laravel 10/11) and upcoming versions.
  • Risk: Older Laravel projects (e.g., Laravel 8 with PHP 7.4) may require polyfills or version pinning.
• False Positives/Negatives:
  • Coverage reports for root directories or files may have edge cases (fixed in v0.2.5), but untested in large Laravel monorepos.
  • Workaround: Test in a staging environment with a representative codebase before full adoption.
• Performance Overhead:
  • Minimal runtime impact (CLI tool), but parsing large coverage files (e.g., 10K+ lines) could slow CI pipelines.
  • Mitigation: Cache coverage reports or run in parallel with other tests.

Key Questions

1. CI/CD Strategy:
   • Should this replace or supplement existing coverage tools (e.g., php-coveralls, sonar-scanner)?
   • How will failures be communicated (e.g., CI badges, Slack alerts)?
2. Threshold Alignment:
   • Should thresholds vary by environment (e.g., 80% for dev, 95% for production)?
   • How will legacy code (e.g., third-party libraries) be excluded?
3. Developer Experience:
   • Should the tool be wrapped in an Artisan command for Laravel-specific workflows?
   • How will teams debug coverage failures (e.g., list of non-compliant files)?
4. Long-Term Maintenance:
   • Who will monitor updates (e.g., PHPUnit 10 compatibility)?
   • Should this be vendor-locked or promoted to a project-level dependency?

Integration Approach

Stack Fit

• Laravel Ecosystem:
  • PHPUnit: Native support via --coverage-php flag.
  • Symfony Bridge: Compatible with Laravel’s kernel testing (e.g., tests/Feature/).
  • CI Tools: Works with GitHub Actions, GitLab CI, and Laravel Forge.
• Alternatives Considered:
  • php-coveralls: Focuses on external reporting (e.g., Coveralls.io) rather than local enforcement.
  • Infection: Mutation testing, not coverage gating.
  • SonarQube: Overkill for projects needing lightweight coverage checks.
• Recommendation: Use as a pre-commit hook (via Laravel Pint/Husky) or CI gate for fast feedback.

Migration Path

1. Phase 1: Pilot in CI

   • Add to composer.json as a dev dependency:

     "require-dev": {
       "handcraftedinthealps/code-coverage-checker": "^0.2.9"
     }
     

   • Update phpunit.xml to generate coverage:

     <phpunit>
       <coverage>
         <include>
           <directory>./app</directory>
         </include>
         <exclude>
           <directory>./vendor</directory>
         </exclude>
         <report>php</report>
       </coverage>
     </phpunit>
     

   • Add CI step (example for GitHub Actions):

     - name: Check coverage
       run: vendor/bin/code-coverage-checker coverage.clover line 90
     

2. Phase 2: Local Enforcement

   • Wrap in a custom Artisan command (e.g., php artisan coverage:check) for local development:

     // app/Console/Commands/CheckCoverage.php
     use Symfony\Component\Process\Process;
     
     public function handle()
     {
         $process = new Process(['phpunit', '--coverage-php=coverage.clover']);
         $process->run();
         $this->call('vendor:publish', ['--provider' => 'HandcraftedInTheAlps\CodeCoverageChecker\CodeCoverageCheckerServiceProvider']);
         $this->info('Running coverage check...');
         $exitCode = shell_exec('vendor/bin/code-coverage-checker coverage.clover line 90');
         if ($exitCode !== 0) exit(1);
     }
     

   • Register in app/Console/Kernel.php:

     protected $commands = [
         Commands\CheckCoverage::class,
     ];
     

3. Phase 3: Granular Thresholds

   • Use directory filtering to enforce higher thresholds for app/ and lower for tests/:

     vendor/bin/code-coverage-checker coverage.clover line 95 app/ tests/Unit/
     

Compatibility

• Laravel Versions:
  • LTS (10.x/11.x): Full compatibility (PHP 8.1+, PHPUnit 9+).
  • Older (8.x/9.x): May require PHPUnit 8.x polyfills or downgrading the package.
• Testing Frameworks:
  • Works with Pest (via PHPUnit under the hood) but may need configuration adjustments.
  • Dusk/BrowserKit: Coverage reports may exclude JavaScript-heavy tests; exclude tests/Browser/ from checks.
• Monorepos:
  • Test with Lumen or Laravel Octane to ensure coverage reports include all modules.

Sequencing

1. Pre-requisites:
   • Ensure PHPUnit is configured to generate coverage reports (phpunit.xml).
   • Verify CI environment has vendor/bin/ in PATH.
2. Order of Operations:
   • Run tests with coverage → Generate report → Run checker → Fail build if thresholds aren’t met.
3. Parallelization:
   • Run coverage checks in parallel with other CI jobs (e.g., security scans) to avoid bottlenecks.

Operational Impact

Maintenance

• Dependency Updates:
  • Monitor for PHPUnit 10 or Symfony 9 compatibility (package is actively maintained).
  • Pin version in composer.json to avoid surprises:

    "handcraftedinthealps/code-coverage-checker": "0.2.9"
    

• Configuration Drift:
  • Centralize thresholds in .env or config/coverage.php to avoid hardcoded CLI values:

    // config/coverage.php
    return [
        'thresholds' => [
            'line' => env('COVERAGE_THRESHOLD_LINE', 90),
            'directories' => ['app/', 'src/'],
        ],
    ];
    

• Tooling:
  • Add to composer.json scripts for local testing:

    "scripts": {
      "test:coverage": "phpunit --coverage-php=coverage.clover && vendor/bin/code-coverage-checker coverage.clover line $(config:coverage.thresholds.line