Phpunit Slow Test Detector Laravel Package

Technical Evaluation

Architecture Fit

• Pros:
  • Non-intrusive: Integrates seamlessly as a PHPUnit extension without modifying core test logic, aligning with Laravel’s reliance on PHPUnit for testing.
  • Configurable: Allows fine-grained control over thresholds (maximum-duration, maximum-count) and output behavior (stderr), which is critical for CI/CD pipelines and large test suites.
  • Version Agnostic: Supports PHPUnit 6.5–13.x, ensuring compatibility with Laravel’s evolving PHPUnit dependencies (e.g., Laravel Pint, Pest integration).
  • Performance Focus: Directly addresses a pain point in CI/CD—slow tests—without requiring manual instrumentation or external tools (e.g., custom scripts).
• Cons:
  • No Native Laravel Integration: Requires manual configuration in phpunit.xml, which may not align with Laravel’s convention-over-configuration philosophy (e.g., phpunit.xml.dist).
  • Output Format: Raw CLI output may need parsing for integration with Laravel’s logging (e.g., Monolog) or CI tools (e.g., GitHub Actions annotations).

Integration Feasibility

• Laravel Stack Fit:
  • PHPUnit: Laravel uses PHPUnit as its default testing framework (via phpunit/phpunit package), making this a drop-in solution.
  • CI/CD: Ideal for pipelines where test execution time is a bottleneck (e.g., GitHub Actions, GitLab CI).
  • Testing Tools: Complements Laravel-specific tools like Pest (which uses PHPUnit under the hood) or custom test suites.
• Dependencies:
  • No Conflicts: MIT-licensed and dependency-light (only requires PHPUnit), posing no risk to Laravel’s ecosystem.
  • Phar Option: Useful for environments where composer is unavailable (e.g., Docker containers).

Technical Risk

• Low:
  • Stability: Actively maintained (releases as recent as 2026), with CI/CD workflows for integration/merge testing.
  • Backward Compatibility: Supports PHPUnit 6.5–13.x, covering Laravel’s typical range (e.g., Laravel 9+ uses PHPUnit 9.x/10.x).
• Mitigations:
  • Testing: Validate with Laravel’s test suite (e.g., php artisan test) to ensure no edge cases (e.g., parallel tests, custom listeners).
  • Fallback: If integration fails, revert to manual time tracking (e.g., microtime() in tests) or external tools like phpunit-speed-tester.

Key Questions

1. Threshold Tuning:
   • What maximum-duration (default: 500ms) aligns with our CI/CD SLAs? (e.g., 1s for unit tests, 5s for integration tests).
   • Should thresholds be environment-specific (e.g., stricter in CI than local dev)?
2. Output Handling:
   • How to integrate stderr output into Laravel’s logging (e.g., Monolog) or CI tooling (e.g., GitHub Actions annotations)?
   • Should slow tests trigger CI failures (requires custom exit code handling)?
3. Performance Impact:
   • Does the extension add measurable overhead to test execution? (Benchmark with/without the extension.)
4. Toolchain Integration:
   • How to surface slow tests in Laravel’s IDE (e.g., PHPStorm) or test reporting tools (e.g., Allure)?
5. Parallel Tests:
   • Does the extension support PHPUnit’s parallel test execution (introduced in v9.x)? If not, is a workaround needed?

Integration Approach

Stack Fit

• Primary Use Case:
  • CI/CD Optimization: Identify and address slow tests in Laravel’s test suite to reduce pipeline time.
  • Developer Feedback: Highlight slow tests locally to encourage refactoring (e.g., mocking external APIs).
• Secondary Use Cases:
  • Test Suite Health: Monitor regression in test performance over time.
  • Onboarding: Educate new developers on test performance expectations.

Migration Path

1. Assessment Phase:
   • Audit current PHPUnit version (phpunit/phpunit in composer.json).
   • Benchmark baseline test execution time (e.g., php artisan test --stop-on-failure).
2. Integration:
   • Option A (Composer):

     composer require --dev ergebnis/phpunit-slow-test-detector
     

     Update phpunit.xml with the extension configuration (see README for version-specific syntax).
   • Option B (Phar): Download the Phar and configure extensionsDirectory in phpunit.xml.
   • Configuration: Start with defaults (maximum-duration=500), then adjust based on CI/CD feedback.
3. Validation:
   • Run tests locally and in CI to verify output format and thresholds.
   • Compare execution time with/without the extension (should be negligible overhead).
4. Automation:
   • Add a CI step to fail builds if slow tests exceed a threshold (e.g., via custom script parsing stderr).

Compatibility

• Laravel-Specific Considerations:
  • Artisan Test Command: Ensure php artisan test (which uses phpunit.xml) picks up the extension.
  • Pest Integration: If using Pest, verify compatibility (Pest uses PHPUnit under the hood, so this should work).
  • Custom Test Suites: Confirm the extension works with Laravel’s TestCase and custom listeners.
• Edge Cases:
  • Parallel Tests: Test with --parallel flag if using PHPUnit 9.x+.
  • Database Transactions: Slow tests involving DB transactions may need separate handling (e.g., refreshDatabase() in Laravel).

Sequencing

1. Phase 1 (Local Dev):
   • Install and configure the extension locally.
   • Run tests with default thresholds to identify slow tests.
2. Phase 2 (CI/CD):
   • Integrate into CI pipeline (e.g., GitHub Actions) with adjusted thresholds.
   • Add a step to parse slow test output and annotate PRs (e.g., GitHub Actions ::warning).
3. Phase 3 (Optimization):
   • Refactor slow tests (e.g., add mocks, reduce assertions).
   • Tighten thresholds iteratively (e.g., reduce maximum-duration by 10% weekly).

Operational Impact

Maintenance

• Proactive:
  • Configuration Management: Maintain version-specific phpunit.xml snippets for different Laravel/PHPUnit versions.
  • Threshold Updates: Periodically review and adjust maximum-duration based on test suite evolution.
• Reactive:
  • False Positives: Monitor for tests incorrectly flagged as slow (e.g., due to external dependencies like APIs).
  • Extension Updates: Watch for breaking changes in ergebnis/phpunit-slow-test-detector (e.g., PHPUnit 14.x support).

Support

• Developer Onboarding:
  • Document the extension’s purpose and thresholds in CONTRIBUTING.md.
  • Provide a template for phpunit.xml configuration.
• Troubleshooting:
  • Common issues:
    • Extension not loading (check bootstrap path in phpunit.xml).
    • Output not appearing (verify PHPUnit version compatibility).
  • Debugging: Run PHPUnit with --debug to inspect extension loading.

Scaling

• Test Suite Growth:
  • Performance: The extension’s overhead is minimal (measured in milliseconds), so it scales with test suite size.
  • Thresholds: Adjust maximum-duration dynamically (e.g., higher for integration tests).
• Distributed Testing:
  • If using parallel tests, ensure the extension works with PHPUnit’s --parallel flag (tested in v9.x+).

Failure Modes

Ramp-Up

• Developer Adoption:
  • Training: Add a section to the team’s testing documentation.
  • Incentives: Tie slow test reduction to performance metrics (e.g., CI time).
• CI/CD Integration:
  • GitHub Actions Example:

    - name: Run PHPUnit with slow test detection
      run: php artisan test
    - name: Parse slow tests
      run: |
        SLOW_TESTS=$(php artisan test 2>&1 | grep -E "Duration.*Test")
        if [ -n "$SLOW_TESTS" ]; then
          echo "$S