Phpunit Slow Test Detector Laravel Package

Getting Started

Minimal Setup

1. Install the package:

   composer require --dev ergebnis/phpunit-slow-test-detector
   

2. Configure phpunit.xml: Add the extension to your PHPUnit config based on your PHPUnit version (see README for version-specific instructions). Example for PHPUnit 10+:

   <phpunit ...>
       <extensions>
           <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
       </extensions>
   </phpunit>
   

3. Run tests:

   ./vendor/bin/phpunit
   

   The extension will now report slow tests (default threshold: 500ms).

First Use Case: Identify Slow Tests

Run your test suite and observe the output. The extension lists tests exceeding the default maximum-duration (500ms) in a table format. Example:

Detected 3 tests where the duration exceeded the global maximum duration (0.500).
# Duration Test
1    0.800 App\Tests\Feature\SlowApiTest::testLongRunningEndpoint
2    0.650 App\Tests\Unit\DatabaseSeederTest::testLargeDatasetLoad
3    0.550 App\Tests\Integration\PaymentGatewayTest::testRetryLogic

Implementation Patterns

1. Integration with Laravel’s Test Workflow

• Laravel’s phpunit.xml: Place the extension config in phpunit.xml (located in the project root or tests/).
• Custom Test Suites: Use the maximum-duration parameter to set stricter thresholds for specific suites:

  <phpunit>
      <extensions>
          <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
      </extensions>
      <parameters>
          <parameter name="maximum-duration" value="200" type="int"/>
      </parameters>
      <testsuites>
          <testsuite name="unit">
              <directory>tests/Unit</directory>
              <parameter name="maximum-duration" value="100" type="int"/>
          </testsuite>
      </testsuites>
  </phpunit>
  

• CI/CD Pipelines: Fail builds if slow tests exceed thresholds by combining with PHPUnit’s --exit-code flag:

  ./vendor/bin/phpunit --exit-code 1
  

2. Dynamic Thresholds

• Environment-Based Config: Use Laravel’s .env to set thresholds dynamically:

  <parameter name="maximum-duration" value="{env('TEST_SLOW_THRESHOLD')}" type="int"/>
  

  In .env:

  TEST_SLOW_THRESHOLD=300  # 300ms
  

• Feature Flags: Toggle slow test detection in CI vs. local:

  # Local (verbose)
  ./vendor/bin/phpunit --configuration phpunit.local.xml
  
  # CI (strict)
  ./vendor/bin/phpunit --configuration phpunit.ci.xml
  

3. Extending Functionality

• Custom Output: Override the default reporter by extending the Extension class:

  namespace App\Tests\Extensions;
  
  use Ergebnis\PHPUnit\SlowTestDetector\Extension as BaseExtension;
  
  class CustomSlowTestDetector extends BaseExtension
  {
      public function report(Report $report)
      {
          // Custom logic (e.g., Slack notifications)
          parent::report($report);
      }
  }
  

  Update phpunit.xml:

  <extensions>
      <bootstrap class="App\Tests\Extensions\CustomSlowTestDetector"/>
  </extensions>
  

4. Parallel Testing

• Xdebug Compatibility: Disable Xdebug for faster tests (add to phpunit.xml):

  <php>
      <server name="XDEBUG_MODE" value="off"/>
  </php>
  

• Dockerized Tests: Run tests in parallel containers with adjusted thresholds:

  docker-compose run --rm -e TEST_SLOW_THRESHOLD=1000 tests ./vendor/bin/phpunit
  

Gotchas and Tips

Pitfalls

1. Version Mismatches:

   • Ensure the extension’s PHPUnit version compatibility matches your project’s (e.g., PHPUnit 10+ uses bootstrap instead of extension).
   • Fix: Check the compatibility table and update phpunit.xml.

2. False Positives:

   • Tests with slow database queries or external API calls may trigger false alarms.
   • Fix: Exclude known slow tests using PHPUnit’s @group and filter in phpunit.xml:

     <testsuites>
         <testsuite name="fast">
             <directory>tests/Unit</directory>
             <exclude>@slow</exclude>
         </testsuite>
     </testsuites>
     

3. Performance Overhead:

   • The extension adds minimal overhead (~1-2ms per test), but excessive thresholds (e.g., maximum-duration=10000) may hide performance issues.
   • Tip: Start with conservative thresholds (e.g., 200ms for unit tests, 500ms for integration).

Debugging

1. Silent Failures:

   • If no slow tests are reported but you suspect issues, verify the extension is loaded by checking PHPUnit’s output for:

     SlowTestDetector: Initialized with max duration=500ms
     

   • Debug: Run with --debug:

     ./vendor/bin/phpunit --debug
     

2. Configuration Errors:

   • Invalid maximum-duration values (e.g., strings) will be ignored. Use type="int" in phpunit.xml:

     <parameter name="maximum-duration" value="500" type="int"/>
     

Tips

1. CI-Specific Thresholds:

   • Use stricter thresholds in CI to catch regressions:

     <!-- phpunit.ci.xml -->
     <parameter name="maximum-duration" value="200" type="int"/>
     

2. Visual Feedback:

   • Combine with PHPUnit’s --colors for better visibility:

     ./vendor/bin/phpunit --colors=always
     

3. Exclude Flaky Tests:

   • Temporarily exclude unstable tests from slow test detection:

     <extensions>
         <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension">
             <parameter name="exclude" value="App\Tests\FlakyTest"/>
         </bootstrap>
     </extensions>
     

4. Laravel Artisan Alias:

   • Add a custom Artisan command for quick access:

     // app/Console/Commands/TestSlow.php
     namespace App\Console\Commands;
     
     use Illuminate\Console\Command;
     
     class TestSlow extends Command
     {
         protected $signature = 'test:slow';
         protected $description = 'Run tests with slow test detection';
     
         public function handle()
         {
             $this->call('phpunit', ['--configuration' => 'phpunit.xml']);
         }
     }
     

   • Run with:

     php artisan test:slow
     

5. GitHub Actions Integration:

   • Fail the build if slow tests exceed thresholds:

     - name: Run tests
       run: ./vendor/bin/phpunit --exit-code 1