Phpunit Slow Test Detector Laravel Package

Technical Evaluation

Architecture Fit

• Pros:

  • Non-intrusive: Integrates seamlessly as a PHPUnit extension without modifying test logic, aligning with Laravel’s test-first philosophy.
  • Configurable: Custom thresholds (maximum-duration, maximum-count) allow alignment with CI/CD pipelines (e.g., flagging tests >2s as slow).
  • Version Agnostic: Supports PHPUnit 6.5–13.x, ensuring compatibility with Laravel’s evolving stack (PHPUnit 9.x+ is standard in Laravel 9+).
  • Output Clarity: Structured slow-test reporting aids debugging and CI/CD monitoring (e.g., Slack/email alerts for flaky tests).

• Cons:

  • No Native Laravel Integration: Requires manual PHPUnit config updates (e.g., phpunit.xml), which may conflict with Laravel’s phpunit.xml.dist or PestPHP adoption.
  • Limited Visual Feedback: Output is CLI-only; no direct integration with Laravel’s Horizon/Forge for real-time dashboards.

Integration Feasibility

• Low Risk: Minimal changes required—only phpunit.xml modifications and composer require.
• Dependencies:
  • PHPUnit: Must be ≥6.5 (Laravel 5.8+ meets this).
  • PHP 8.1+: Future-proof for Laravel 10+ (current LTS).
• Conflict Potential:
  • PestPHP: If using Pest, this tool is redundant (Pest has built-in --min-time flag). Requires evaluation of trade-offs (e.g., PHPUnit’s richer ecosystem vs. Pest’s simplicity).
  • Custom Test Suites: May need adjustments if tests use non-standard PHPUnit extensions.

Technical Risk

Key Questions

1. CI/CD Pipeline Impact:

   • How will slow-test alerts integrate with existing monitoring (e.g., GitHub Actions, Jenkins)?
   • Should failures trigger build breaks or just warnings?

2. Toolchain Alignment:

   • Is PHPUnit the primary test runner, or should we prioritize PestPHP’s native timing features?

3. Threshold Strategy:

   • Should thresholds be project-wide (e.g., maximum-duration="2000") or environment-specific (e.g., CI vs. local)?

4. Maintenance:

   • Who owns phpunit.xml updates (DevOps, QA, or TPM)?

Integration Approach

Stack Fit

• Laravel Compatibility:

  • PHPUnit: Native support in Laravel (via phpunit.xml.dist). No framework-level conflicts.
  • PestPHP: Partial overlap; assess whether Pest’s --min-time suffices before adopting this tool.
  • CI Tools: Works with GitHub Actions, CircleCI, etc., via CLI output parsing (e.g., grep "slow test").

• Alternatives Evaluated:

  • PestPHP --min-time: Simpler but lacks PHPUnit’s extensibility.
  • Custom Scripts: Higher maintenance (e.g., parsing phpunit --log-junit).
  • Xdebug Profiler: Overkill for CI; better for local debugging.

Migration Path

1. Assessment Phase:

   • Audit current PHPUnit/Pest usage (composer.json, CI configs).
   • Measure baseline test durations (e.g., phpunit --log-junit | jq '.testcases[] | {name, time}').

2. Pilot:

   • Install in a feature branch:

     composer require --dev ergebnis/phpunit-slow-test-detector
     

   • Update phpunit.xml with:

     <extensions>
       <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
     </extensions>
     <parameters>
       <parameter name="maximum-duration" value="1000"/> <!-- 1s threshold -->
     </parameters>
     

   • Test in CI with a dry run (e.g., phpunit --stop-on-failure).

3. Rollout:

   • Phase 1: Enable in CI only (monitor false positives).
   • Phase 2: Add to local dev environments (document thresholds in README.md).
   • Phase 3: Integrate with alerts (e.g., Slack via phpunit | grep "slow test" | curl -X POST -d @- SLACK_WEBHOOK).

Compatibility

Sequencing

1. Pre-requisite: Ensure PHPUnit ≥6.5 (Laravel 5.8+ meets this).
2. Config Update: Modify phpunit.xml (or create a new config for CI).
3. Threshold Tuning: Start with conservative values (e.g., 500ms) and adjust based on CI data.
4. Alerting: Last step—integrate with monitoring tools.

Operational Impact

Maintenance

• Proactive:
  • Config Management: Version-control phpunit.xml changes (e.g., Git diffs for thresholds).
  • Dependency Updates: Monitor PHPUnit major versions (e.g., upgrade to v13.x when Laravel supports it).
• Reactive:
  • False Positives: Maintain a KNOWN_SLOW_TESTS.md to document intentional slow tests (e.g., integration tests).
  • Threshold Adjustments: Quarterly review of maximum-duration based on CI trends.

Support

• Developer Onboarding:
  • Add a TESTING.md section:

    "Slow tests are flagged if they exceed 1s. Use @slow annotations for intentional slow tests (see PHPUnit docs)."

  • Provide a CLI snippet for local testing:

    phpunit --configuration phpunit-slow.xml  # Uses custom slow-test config
    

• CI Troubleshooting:
  • Log slow-test reports to a file (phpunit > slow-tests.log) for debugging.
  • Template for alert messages:

    :warning: Slow tests detected in <branch>:
    <phpunit-slow-test-detector-output>
    

Scaling

• Performance:
  • Impact: Minimal (<3% overhead per test suite). Benchmark with:

    hyperfine 'phpunit' 'phpunit --extensions slow-test-detector'
    

  • CI Parallelization: Works with PHPUnit’s --parallel flag (no conflicts).
• Test Suite Growth:
  • Threshold Scaling: Use relative thresholds (e.g., maximum-duration="avg_duration * 1.5") via custom PHP scripts.

Failure Modes

Ramp-Up

• Time Estimate: 2–4 hours (pilot + CI integration).
• Key Milestones:
  1. Day 1: Install and configure locally.
  2. Day 2: Run in CI with --stop-on-failure=false.
  3. Week 1: Integrate with alerts and document thresholds.
• Training:
  • For Developers: 15-minute demo on interpreting slow-test output.
  • For QA/DevOps: Guide on adjusting thresholds and suppressing false positives.
• Metrics for Success:
  • Reduction in Flaky Tests: Track via GitHub issues labeled flaky.
  • CI Speed: Measure median test suite duration pre/post-adoption.