Multi Tester Laravel Package

Getting Started

Minimal Steps

1. Install the Package Add multi-tester as a dev dependency in your Laravel package:

   composer require kylekatarnls/multi-tester --dev
   

2. Initialize Configuration Create a .multi-tester.yml file in your project root. Example for testing against Laravel core and a common auth package:

   laravel/framework:
     version: ^10.0
   spatie/laravel-permission:
     version: ^5.0
   

3. First Test Run Execute the command to test your package against the configured dependencies:

   vendor/bin/multi-tester
   

4. Add a New Dependency To add a new package dynamically (e.g., nesbot/carbon):

   vendor/bin/multi-tester --add=nesbot/carbon
   

First Use Case: Testing Laravel Package Changes

• Scenario: You’ve updated a Laravel package (e.g., a custom auth provider) and want to ensure it doesn’t break Laravel core or other dependent packages.
• Workflow:
  1. Update your package’s composer.json and push changes.
  2. Run multi-tester to validate against Laravel’s test suite and other critical dependencies.
  3. Fix any failures reported by the dependent packages’ test suites.

Implementation Patterns

Core Workflows

1. Dependency Validation in CI/CD

• Pattern: Integrate multi-tester into your Laravel package’s CI pipeline (e.g., GitHub Actions or GitLab CI) to run cross-project tests on every push.
• Example GitHub Actions Workflow:

  jobs:
    test:
      runs-on: ubuntu-latest
      steps:
        - uses: actions/checkout@v4
        - run: composer install
        - run: vendor/bin/multi-tester
  

• Tip: Use the --stop-on-failure flag to halt the pipeline if any dependent test fails.

2. Testing Multiple Versions

• Pattern: Test your Laravel package against multiple versions of a dependency (e.g., Laravel 9.x, 10.x, and 11.x).
• Config Example:

  laravel/framework:9.*:
    script: vendor/bin/phpunit --testsuite=Unit
  laravel/framework:10.*:
    script: vendor/bin/phpunit --testsuite=Unit
  laravel/framework:11.*:
    script: vendor/bin/phpunit --testsuite=Unit
  

• Use Case: Ensures backward compatibility for Laravel packages targeting older versions.

3. Leveraging Travis CI Commands

• Pattern: Use travis shortcuts in your config to inherit the dependency’s CI setup.
• Config Example:

  spatie/laravel-permission:
    travis  # Automatically pulls `install` and `script` from .travis.yml
  

• Benefit: Avoids manual configuration for dependencies with well-defined CI workflows.

4. Isolated Testing with Custom Scripts

• Pattern: Override default test commands for specific dependencies (e.g., running only critical tests).
• Config Example:

  illuminate/support:
    install: composer install --prefer-dist --no-interaction
    script: vendor/bin/phpunit --filter=AuthTest
  

5. Dynamic Configuration with CLI

• Pattern: Use the --add flag to dynamically include dependencies during development.
• Example:

  vendor/bin/multi-tester --add=laravel/valet --version=^4.0
  

Integration Tips

• Laravel-Specific:
  • For Laravel packages, prioritize testing against laravel/framework and illuminate/* dependencies.
  • Use --version to test against specific Laravel minor versions (e.g., laravel/framework:10.2.*).
• Performance:
  • Cache dependent repositories locally to avoid repeated clones. Use success_only: true to test against the last known good commit.
  • Example:

    config:
      success_only: true
    

• Debugging:
  • Use -v or --verbose to diagnose issues (e.g., failed clones or test runs).
  • Example:

    vendor/bin/multi-tester -v
    

Gotchas and Tips

Pitfalls

1. Composer Autoload Conflicts

   • Issue: Your package’s autoload may conflict with the dependent package’s autoload, causing test failures.
   • Fix: Use composer dump-autoload in the install script to regenerate autoload files.

     my/package:
       install: composer dump-autoload && composer install
     

2. Git Clone Failures

   • Issue: Dependencies with private repositories or complex Git histories may fail to clone.
   • Fix: Specify a custom clone command or use SSH URLs if GitHub Actions is involved.

     private/package:
       clone: git clone git@github.com:org/private-package.git .
     

3. PHP Version Mismatches

   • Issue: Dependencies may require specific PHP versions, causing tests to fail.
   • Fix: Use composer require php:^8.1 in your package’s composer.json or configure multi-tester to use a compatible PHP version in CI.

4. Travis CI Shortcut Limitations

   • Issue: The travis shortcut may not work if the dependency’s .travis.yml is malformed or missing.
   • Fix: Manually specify install and script if the shortcut fails.

5. Working Directory Isolation

   • Issue: Tests may fail due to shared state between runs (e.g., cached files).
   • Fix: Use success_only: true or clean the working directory between runs:

     config:
       clean_after: true
     

6. Packagist API Rate Limits

   • Issue: Frequent API calls to Packagist may hit rate limits.
   • Fix: Cache the Packagist response locally or use libraries.io as a fallback (configured automatically).

Debugging Tips

• Check Logs: Use -v to see detailed output for each step (clone, install, test).
• Isolate Failures: Test one dependency at a time by commenting out others in .multi-tester.yml.
• Manual Overrides: Temporarily override install or script to debug specific issues:

  problematic/package:
    install: composer install --verbose
    script: vendor/bin/phpunit -d
  

Extension Points

1. Custom Commands

   • Extend the config to include pre/post-test hooks (e.g., database migrations or cache clearing):

     my/package:
       pre_test: php artisan migrate:fresh
       script: vendor/bin/phpunit
     

2. Parallel Testing

   • Use xargs or parallel to run multi-tester against multiple configs in parallel:

     find . -name "*.multi-tester.yml" -exec vendor/bin/multi-tester {} \;
     

3. GitHub Actions Matrix

   • Combine multi-tester with GitHub Actions’ matrix strategy to test against multiple Laravel versions:

     strategy:
       matrix:
         laravel: [9.*, 10.*, 11.*]
     jobs:
       test:
         runs-on: ubuntu-latest
         steps:
           - run: vendor/bin/multi-tester --config=.multi-tester-${{ matrix.laravel }}.yml
     

4. Custom Config Files

   • Use multiple config files for different environments (e.g., dev.multi-tester.yml vs. prod.multi-tester.yml):

     vendor/bin/multi-tester path/to/config.yml
     

Laravel-Specific Quirks

• Laravel Artisan Commands: If tests require Artisan commands, ensure the APP_ENV and DB_CONNECTION are set in the dependent package’s environment.
• Service Providers: Some Laravel packages may need providers registered before testing. Add a pre_test script:

  my/package:
    pre_test: php artisan package:discover
    script: vendor/bin/phpunit
  

• Configuration Publishing: If your package publishes config files, ensure the dependent package’s config is updated or mocked during tests.