phpyh/psalm-tester
Run Psalm assertions from .phpt files. Write phpt tests with --FILE-- and --EXPECT/EXPECTF-- for traces and errors, then execute them via a PHPUnit test suite using PsalmTester. Supports per-suite default Psalm args and per-test --ARGS-- overrides.
## Technical Evaluation
### **Architecture Fit**
- **Static Analysis Integration**: The package remains a strong fit for Laravel/PHP projects leveraging **Psalm**, now with a **test-friendly wrapper** that aligns with Laravel’s **PHPUnit/Pest** ecosystem. The initial release introduces **assertion-based validation** (`assertNoErrors()`, `assertErrorsMatchBaseline()`), enabling seamless integration into test suites (e.g., `it('has no Psalm errors', fn() => PsalmTester::assertNoErrors())`). This reinforces **shift-left quality gates** and **deterministic CI/CD validation**.
- **CI/CD Compatibility**: Designed for **GitHub Actions/GitLab CI**, the package’s **baseline comparison** approach ensures **regression detection** without manual review of raw Psalm logs. The new `assertErrorsMatchBaseline()` feature is particularly valuable for **controlled adoption** in CI pipelines.
- **Lightweight Abstraction**: The package avoids reinventing Psalm’s core logic while adding **test assertions** and **CI-friendly hooks**. The initial release is minimal but sets the foundation for future extensions (e.g., custom rules, parallelization, or IDE integration).
- **Laravel-Specific Enhancements**: While the package does not yet explicitly handle **Laravel-specific Psalm configurations** (e.g., Facade stubs, `app()` helper), its **agnostic design** allows integration with existing Laravel Psalm plugins (e.g., `psalm-plugin-laravel`). This makes it adaptable to Laravel’s dynamic features, though manual tuning may still be required.
### **Integration Feasibility**
- **Minimal Boilerplate**: Requires only `composer require phyp/psalm-tester` + basic Psalm setup (`psalm.json`). No Laravel-specific configuration is mandatory, though **Laravel-specific Psalm plugins** may need explicit integration for optimal results.
- **New Feature: Baseline Assertions**: The `assertErrorsMatchBaseline()` method is a **game-changer for CI/CD**, reducing the need for manual baseline updates in early adoption phases. This aligns with Laravel’s **test-driven workflows** and **automated quality gates**.
- **Artisan and Event Hooks**: Can integrate with:
- **Artisan Commands** (e.g., `php artisan psalm:test`).
- **Test Events** (e.g., run Psalm checks post-migration or pre-deploy).
- **Laravel Forge/Envoyer** for deployment-time validation.
- **Psalm Version Support**: Works with **Psalm 4.x/5.x** but lacks explicit **Psalm 6.x support**. This could become a **breaking change risk** if Psalm’s API evolves significantly in future versions.
- **Performance Considerations**: Running Psalm in CI/tests adds **~5–15s per run**, which is mitigated by caching or parallelization. The initial release does not yet support **parallel execution**, which could become a bottleneck for large codebases.
### **Technical Risk**
- **False Positives/Negatives**: Psalm’s output remains **context-dependent**, especially for Laravel’s dynamic features (e.g., **Facades**, **service providers**). The package’s baseline comparison may still require tuning for these cases, though `assertErrorsMatchBaseline()` reduces manual intervention.
- **Performance Overhead**: The lack of **parallelization support** in 0.1.0 could slow down CI pipelines for large codebases. This should be addressed in future releases.
- **Tooling Ecosystem**: Limited adoption (2 stars) may indicate **undocumented edge cases** (e.g., interactions with Laravel’s `app()` helper or **dynamic method calls**). The initial release is **feature-complete but untested in complex scenarios**.
- **Baseline Drift**: While `assertErrorsMatchBaseline()` helps, **manual baseline updates** are still required if Psalm’s error messages change (e.g., new rule versions). No **automated baseline synchronization** tools exist yet.
- **Version Locking**: The initial release does not explicitly declare **Psalm version constraints**, which could lead to **breaking changes** if Psalm’s API evolves. This is a **critical risk** for long-term stability.
- **IDE Conflicts**: No known conflicts with **PHPStorm/VSCode Psalm plugins**, but this remains **untested** in real-world Laravel projects.
### **Key Questions**
1. **Psalm Configuration**: How does the package handle **Laravel-specific Psalm configs** (e.g., `src/` vs. `app/` paths, Facade stubs)? *(Unclear if the initial release addresses this; may require manual `psalm.json` tweaks.)*
2. **Parallelization**: Can it integrate with **Psalm’s `--parallel` flag** or **Laravel’s queue workers** for large codebases? *(Not supported in 0.1.0; high priority for future releases.)*
3. **IDE Sync**: Does it conflict with **PHPStorm/VSCode Psalm plugins** (e.g., duplicate analysis runs)? *(No known conflicts, but untested in Laravel-specific workflows.)*
4. **Custom Rules**: How are **Laravel-specific Psalm rules** (e.g., `NoUnresolvableTypeForParam`) handled in assertions? *(Not explicitly documented in 0.1.0; may require plugin integration.)*
5. **Legacy Code**: What’s the strategy for **gradually adopting** Psalm checks in a pre-existing codebase? *(Baseline assertions help, but no migration tooling exists yet.)*
6. **CI Caching**: Does it support **GitHub Actions caching** or **Psalm’s `--output-format=json`** for faster CI runs? *(No explicit support in 0.1.0; should be added for performance.)*
7. **Psalm Version Support**: Will the package support **Psalm 6.x** when released? *(Undocumented; potential breaking change risk if Psalm’s API evolves.)*
8. **Artisan Integration**: How robust is the `php artisan psalm:test` command? *(Initial release may lack error handling for edge cases; needs validation.)*
9. **Custom Assertions**: Can developers extend the package with **custom Psalm rule assertions**? *(Not documented; may require direct Psalm API usage.)*
10. **Monorepo Support**: How does it handle **multi-project Psalm setups** (e.g., shared stubs, global baselines)? *(No built-in support; may require manual configuration.)*
---
## Integration Approach
### **Stack Fit**
- **Laravel Core**: Ideal for projects using:
- **Psalm** (static analysis).
- **PHPUnit/Pest** (test suites).
- **GitHub Actions/GitLab CI** (CI pipelines).
- **Complementary Tools**:
- **Pest**: Use `PsalmTester` in feature tests (e.g., `it('has no Psalm errors', fn() => PsalmTester::assertNoErrors())`).
- **Laravel Mix/Vite**: Trigger Psalm checks in build pipelines.
- **Deployer**: Run pre-deploy validation via `PsalmTester::run()`.
- **Anti-Patterns**: Avoid using in:
- **Production runtime** (static analysis only).
- **Projects without Psalm** (requires prior setup).
- **Monorepos** (baseline management becomes complex without tooling).
### **Migration Path**
1. **Phase 1: Setup** *(Unchanged)*
- Install Psalm (`composer require vimeo/psalm`) and configure `psalm.json` for Laravel.
- Add `phpyh/psalm-tester` (`composer require phyp/psalm-tester`).
- Create a **baseline file** (e.g., `tests/PsalmBaseline.txt`) for initial output.
2. **Phase 2: Test Integration** *(Updated for 0.1.0)*
- Add a **test trait** (e.g., `uses(PsalmTester::class)`) to relevant test classes.
- Write **assertions** for critical paths:
```php
public function test_no_psalm_errors_in_auth_controller() {
PsalmTester::assertNoErrors(app_path('Http/Controllers/AuthController.php'));
}
```
- Use **`assertErrorsMatchBaseline()`** for controlled regressions:
```php
public function test_psalm_errors_match_baseline() {
PsalmTester::assertErrorsMatchBaseline();
}
```
- **New**: Leverage `assertErrorsMatchBaseline()` to **reduce manual baseline updates** and enable **gradual adoption**.
3. **Phase 3: CI/CD** *(Updated for 0.1.0)*
- Add a **CI job** (e.g., `.github/workflows/psalm.yml`):
```yaml
- name: Psalm Static Analysis
run: vendor/bin/pest --test PsalmTester::run()
```
- Configure **caching** for Psalm’s `stubs/` and `internal/` directories.
- **New**: Use `assertErrorsMatchBaseline()` to **fail builds only on new errors**, not existing ones, during initial adoption.
4. **Phase 4: Enforcement** *(Unchanged)*
- Fail builds on Psalm errors (default behavior).
- Gradually expand coverage (e.g., start with `app/Http/`, then `app/Console/`
How can I help you explore Laravel packages today?