Multi Tester Laravel Package

Technical Evaluation

Architecture Fit

• Laravel/PHP Ecosystem Alignment: Perfect fit for Laravel-centric packages due to Composer-native design. Leverages Laravel’s dependency graph (e.g., illuminate/, spatie/) and PHP’s vendor/ structure, reducing integration friction.
• Modularity: Config-driven approach (*.multi-tester.yml) aligns with Laravel’s modular philosophy (e.g., service providers, packages). Supports versioned testing (e.g., Laravel 9.x vs. 11.x) via SemVer constraints.
• CI/CD Synergy: Designed for Travis CI/GitHub Actions, complementing Laravel’s CI tooling (e.g., phpunit, pest). Reduces pipeline complexity by consolidating cross-project tests into a single job.
• Language Agnostic: While PHP-based, it runs tests for any language/framework (e.g., JavaScript, Python) via Composer, useful for Laravel’s polyglot ecosystems (e.g., Inertia.js, Livewire).

Integration Feasibility

• Composer Dependency: Installs as a dev dependency, non-intrusive to production. No Laravel service provider or Facade required—pure CLI tool.
• Laravel-Specific Quirks:
  • Handles Laravel’s replace key in composer.json (critical for monorepos like laravel/framework).
  • Supports allow-plugins (e.g., Laravel’s laravel/installer).
  • Validates against Laravel’s autoloading (e.g., psr-4 paths in composer.json).
• GitHub Actions/Travix CI: Pre-configured for Laravel’s CI stacks (PHP 8.1–8.3, Laravel 9.x–11.x). Example:

  jobs:
    multi-test:
      runs-on: ubuntu-latest
      steps:
        - uses: actions/checkout@v4
        - run: composer require kylekatarnls/multi-tester --dev
        - run: vendor/bin/multi-tester --config=.multi-tester.laravel.yml
  

Technical Risk

Key Questions for TPM

1. Dependency Scope:
   • Which Laravel packages are critical dependencies (e.g., illuminate/support, spatie/laravel-permission)?
   • Are there internal monorepos (e.g., laravel/) requiring replace-key handling?
2. CI/CD Constraints:
   • What’s the max allowed CI runtime for cross-project tests? (Target: <30 mins.)
   • Can we parallelize tests across PHP/Laravel versions?
3. Laravel Version Support:
   • Should we test against all supported Laravel versions (9.x–11.x) or focus on LTS?
   • How to handle PHP 8.3+ attributes (e.g., #[\Override]) in tested packages?
4. Failure Handling:
   • Should multi-tester block merges (GitHub PR checks) or warn only?
   • Need detailed diffs for test failures (e.g., phpunit --debug)?
5. Maintenance:
   • Who owns .multi-tester.yml updates? (Devs, QA, or a shared config repo?)
   • How to version-test Laravel’s own packages (e.g., laravel/valet)?

Integration Approach

Stack Fit

• Laravel Core/Ecosystem:
  • Primary Use Case: Validate changes to laravel/framework, illuminate/, or spatie/ packages against downstream users.
  • Example Config:

    # .multi-tester.laravel.yml
    laravel/framework:11.x:
      clone: git clone https://github.com/laravel/framework.git .
      install: composer install --prefer-dist --no-interaction
      script: vendor/bin/phpunit --testsuite=Unit
      stop_on_failure: true
    
    spatie/laravel-permission:5.*:
      travis: true  # Auto-pulls from .travis.yml
    

• Laravel Add-ons:
  • Test service providers, utilities, or Legacy Factories against Laravel’s vendor/ structure.
  • Example: Validate tightenco/ziggy against Laravel 10.x–11.x.
• PHP Libraries:
  • Use for non-Laravel PHP packages with Composer dependencies (e.g., nunomaduro/collision).

Migration Path

1. Pilot Phase:
   • Start with 1–2 critical dependencies (e.g., illuminate/support + spatie/laravel-permission).
   • Use --add to auto-generate .multi-tester.yml:

     vendor/bin/multi-tester --add=illuminate/support
     

2. CI Integration:
   • Add to GitHub Actions (recommended for Laravel):

     - name: Run multi-tester
       if: github.event_name == 'push' || github.event_name == 'pull_request'
       run: vendor/bin/multi-tester --config=.multi-tester.laravel.yml --stop-on-failure
     

   • For Travis CI, use the MULTITEST env var as shown in the README.
3. Gradual Expansion:
   • Add Laravel versions (e.g., laravel/framework:10.*, laravel/framework:11.*).
   • Include PHP versions (8.1–8.3) via php: matrix in CI.
4. Automation:
   • Use composer.json scripts to trigger multi-tester:

     "scripts": {
       "test:multi": "multi-tester --config=.multi-tester.laravel.yml --verbose"
     }
     

Compatibility

Sequencing

1. Pre-Release Workflow:
   • Run multi-tester after phpunit in CI:

     composer test → phpunit → multi-tester
     

2. Post-Merge Validation:
   • Use GitHub PR checks to block merges if tests fail:

     jobs:
       multi-test:
         runs-on: ubuntu-latest
         steps:
           - uses: actions/checkout@v4
           - run: composer install
           - run: vendor/bin/multi-tester --stop-on-failure
     

3. Release Candidate (RC) Phase:
   • Test against all Laravel versions in parallel:

     strategy:
       matrix:
         laravel: [10.x, 11.x]
     

4. Rollback Plan:
   • If multi-tester fails, revert the change and investigate with:

     vendor/bin/multi-tester --verbose --debug
     

Operational Impact

Maintenance

• Configuration Management:
  • Store .multi-tester.yml in the root of the package repo or a shared config repo for multi-package projects.
  • Use templates for common Laravel setups:

    # templates/multi-tester.laravel.yml
    laravel/framework:{{version}}:
      clone: git clone https://github.com/laravel/framework.git .
      install: composer install --prefer-dist --no-interaction
      script: vendor/bin