Psalm Tester Laravel Package

Technical Evaluation

Architecture Fit

• Static Analysis Integration: Fits seamlessly into Laravel/PHP projects leveraging Psalm for type safety, as it standardizes execution and validation of static analysis results.
• Test-Driven Validation: Aligns with Laravel’s growing adoption of PHPUnit/Pest for testing, enabling assertion-based Psalm checks in unit/integration tests.
• CI/CD Compatibility: Designed for GitHub Actions/GitLab CI, where deterministic Psalm output is critical for regression detection.
• Lightweight Abstraction: Avoids reinventing Psalm’s core logic while adding test-friendly wrappers (e.g., PsalmTester::assertNoErrors()).

Integration Feasibility

• Minimal Boilerplate: Requires only composer require + basic config (Psalm’s existing psalm.json remains untouched).
• Laravel-Specific Hooks: Can integrate with:
  • Artisan Commands (e.g., php artisan psalm:test).
  • Test Events (e.g., run Psalm checks post-migration or pre-deploy).
  • Laravel Forge/Envoyer for deployment-time validation.
• Psalm Version Agnostic: Works with any Psalm 4.x/5.x setup, reducing version-locking risks.

Technical Risk

• False Positives/Negatives: Psalm’s output is context-dependent (e.g., autoloading, IDE plugins). The package’s baseline comparison may need tuning for Laravel’s dynamic features (e.g., service providers, Facades).
• Performance Overhead: Running Psalm in CI/tests adds ~5–15s per run (mitigated by caching or parallelization).
• Tooling Ecosystem: Limited adoption (2 stars) may indicate niche use cases or undocumented edge cases (e.g., Laravel’s app() helper introspection).
• Baseline Drift: Manual baseline updates required if Psalm’s error messages change (e.g., new rule versions).

Key Questions

1. Psalm Configuration: How does the package handle Laravel-specific Psalm configs (e.g., src/ vs. app/ paths, Facade stubs)?
2. Parallelization: Can it integrate with Laravel’s queue workers or Psalm’s --parallel flag for large codebases?
3. IDE Sync: Does it conflict with PHPStorm/VSCode Psalm plugins (e.g., duplicate analysis runs)?
4. Custom Rules: How are Laravel-specific Psalm rules (e.g., NoUnresolvableTypeForParam) handled in assertions?
5. Legacy Code: What’s the strategy for gradually adopting Psalm checks in a pre-existing codebase?
6. CI Caching: Does it support GitHub Actions caching or Psalm’s --output-format=json for faster CI runs?

Integration Approach

Stack Fit

• Laravel Core: Ideal for projects using:
  • Psalm (static analysis).
  • PHPUnit/Pest (test suites).
  • GitHub Actions/GitLab CI (CI pipelines).
• Complementary Tools:
  • Pest: Use PsalmTester in feature tests (e.g., it('has no Psalm errors', fn() => PsalmTester::assertNoErrors())).
  • Laravel Mix/Vite: Trigger Psalm checks in build pipelines.
  • Deployer: Run pre-deploy validation via PsalmTester::run().
• Anti-Patterns: Avoid using in:
  • Production runtime (static analysis only).
  • Projects without Psalm (requires prior setup).

Migration Path

1. Phase 1: Setup

   • Install Psalm (composer require vimeo/psalm) and configure psalm.json for Laravel.
   • Add phpyh/psalm-tester (composer require phpyh/psalm-tester).
   • Create a baseline file (e.g., tests/PsalmBaseline.txt) for initial output.

2. Phase 2: Test Integration

   • Add a test trait (e.g., uses(PsalmTester::class)) to relevant test classes.
   • Write assertions for critical paths:

     public function test_no_psalm_errors_in_auth_controller() {
         PsalmTester::assertNoErrors(app_path('Http/Controllers/AuthController.php'));
     }
     

   • Use assertErrorsMatchBaseline() for controlled regressions.

3. Phase 3: CI/CD

   • Add a CI job (e.g., .github/workflows/psalm.yml):

     - name: Psalm Static Analysis
       run: vendor/bin/pest --test PsalmTester::run()
     

   • Configure caching for Psalm’s stubs/ and internal/ directories.

4. Phase 4: Enforcement

   • Fail builds on Psalm errors (default behavior).
   • Gradually expand coverage (e.g., start with app/Http/, then app/Console/).

Compatibility

• Psalm 4.x/5.x: Tested with latest versions; check for breaking changes in composer.json.
• Laravel 8+: Assumes modern autoloading (psr-4). Older versions may need psalm.json tweaks.
• Custom Psalm Plugins: Works with plugins like psalm-plugin-laravel, but baseline files may need updates.
• Monorepos: Requires per-project baselines if using multiple Laravel apps.

Sequencing

Operational Impact

Maintenance

• Baseline Management:
  • Pros: Catches regressions early; self-documenting.
  • Cons: Requires manual updates when Psalm rules change (e.g., new deprecation warnings).
  • Mitigation: Use PsalmTester::assertErrorsMatchBaseline() with semantic versioning for baselines.
• Psalm Updates:
  • Minor updates (e.g., Psalm 5.0 → 5.1) may require baseline regeneration.
  • Major updates may need rule configuration adjustments (e.g., stricter type checking).
• Laravel Updates:
  • New Laravel versions (e.g., 10.x) may introduce Psalm-compatible changes (e.g., new Facades), requiring baseline updates.

Support

• Debugging:
  • Psalm errors are verbose but opaque; psalm-tester adds a layer of abstraction.
  • Workaround: Use PsalmTester::run()->getOutput() to inspect raw Psalm logs.
• Community:
  • Limited adoption (2 stars) → self-support for now.
  • Alternatives: roave/security-advisories (for security-focused checks) or custom scripts.
• Onboarding:
  • Developers need to understand:
    • Why Psalm errors fail CI.
    • How to update baselines without false positives.

Scaling

• Performance:
  • Single File: ~1–3s per file (Psalm’s bottleneck).
  • Full Codebase: ~30–90s (mitigate with parallel runs or CI caching).
  • Laravel-Specific: Exclude vendor/, node_modules/ from analysis.
• Team Size:
  • Small Teams: Low overhead; manual baseline updates.
  • Large Teams: Consider dedicated Psalm maintainers or automated baseline tools (e.g., psalm --init).
• Microservices:
  • Run per-service with isolated baselines.
  • Use Git submodules or monorepo tools (e.g., Nx) to manage Psalm configs.

Failure Modes