Psalm Tester Laravel Package

Technical Evaluation

Architecture fit

• Dev-only, zero production footprint. The package is a require --dev dependency that bridges PHPUnit and Psalm: .phpt fixtures carry --FILE-- / --EXPECT-- / --EXPECTF-- sections, PsalmTester shells out to Psalm, and results are asserted through a PHPUnit constraint. Nothing ships to runtime, so there is no blast radius on application code, request lifecycle, or deployment artifacts.
• It sits beside your existing Psalm config rather than replacing it. Default arguments are --no-progress --no-diff --config=src/psalm.xml, overridable globally via PsalmTester::create(defaultArguments: ...) or per fixture via --ARGS--. For a Laravel codebase that means the analysis contract stays in your psalm.xml + psalm-plugin-laravel; the package only adds fixture-level assertions on top.
• The design assumes you are testing Psalm's behavior (inference, plugin output, stub correctness) rather than testing your application's type-cleanliness. That is a meaningfully narrower product than "static analysis in CI," and it changes who the buyer is: plugin/stub/integration maintainers, not typical product teams.

Integration feasibility

• Integration is genuinely small: one composer require --dev, one PHPUnit TestCase (~15 lines for the single-fixture path), fixtures as files. The 0.4.0 line adds PsalmPhptTestCase with phptDirectory(), which collapses the whole thing to fixture discovery + batched run + --filter narrowing — no custom test class plumbing.
• Two hard dependency facts to verify before committing:
  • vimeo/psalm (^6.10 || ^7.0.0-beta16) and composer-runtime-api ^2 are required, not suggested. Any project pinned below Psalm 6.10, or running a Psalm 7 pre-beta-16, will hit a Composer conflict. This is the single most likely first-hour blocker.
  • PHPUnit attributes (#[TestWith], #[DataProvider]) in the documented examples imply PHPUnit 10+.
• The structured run() / runOne() API returning Result with Outcome and Issue objects is the real integration lever: gating, reporting, and trend capture become ordinary code rather than log parsing.

Technical risk

• Adoption signal is near-zero. 1 star, 0 dependents, single maintainer. This is not a validation problem — niche dev tooling often looks like this — but it is a bus-factor and continuity problem. Treat sourcing as "adopt with a fork plan," not "adopt a vendor."
• Pre-1.0 with breaking releases. 0.4.0 removed progress output, rejected --EXPECT*_EXTERNAL--, changed --ARGS-- from replace to append, made withArguments() one-arg-per-parameter, removed the shell, and turned repeated sections into hard errors. Every one of those is a silent-or-loud CI break. The existence of UPGRADING.md is a good sign; the cadence is not.
• Beta upstream dependency. Depending on ^7.0.0-beta16 means your harness version is coupled to Psalm's pre-release track. Psalm 7 stable could shift inference and your expectations simultaneously.
• Correctness hazards baked into the model.
  • --EXPECTF-- wildcards (%d, %s) are convenient and also the easiest way to write an assertion that can never fail meaningfully — an expectation-drift risk that needs review discipline.
  • runBatch() shares one global symbol table across fixtures in an argument group, so duplicate class/function names produce DuplicateClass / DuplicateFunction noise. This is documented, but it is a design constraint teams will hit at scale.
  • Timeout kills the Psalm process and its children on Linux/macOS; on Windows only the Psalm process itself — a real gap if any CI runs Windows.
• Metadata anomaly. The listed last release date (2026-09-30) is in the future relative to a normal review cycle. Verify the actual published version on Packagist before pinning anything.

Key questions

1. Do we actually maintain a Psalm plugin, stub set, or framework integration — or do we only consume Psalm? If the latter, expected value drops sharply and this is probably a no.
2. Which Psalm version does our codebase and psalm-plugin-laravel pin today, and does it satisfy ^6.10 || ^7.0.0-beta16?
3. Is our CI Linux-only? If Windows runners are in scope, do we accept the process-tree limitation as a known gap?
4. Who owns the fixture suite long-term — is there a named engineer who will review EXPECTF wildcards and --XFAIL-- expiry, or will this rot like a baseline file?
5. What is our fork-and-maintain cost if upstream goes quiet for two quarters?
6. What is the exit path — can we drop back to raw Psalm invocations + string assertions without losing meaningful coverage?

Integration Approach

Stack fit

• Best fit: a Laravel/PHP monorepo or plugin repo that already runs PHPUnit and Psalm in the same pipeline. The package slots in as one more TestCase, so no new CI stage, no new runner, no new reporting surface — phpunit --filter becomes the way to run a single type fixture.
• The Laravel-specific angle is explicit in the 0.4.0 notes: psalm-plugin-laravel's type suite (758 fixtures) is the reference workload, and runBatch() exists specifically because booting a full Laravel application per fixture is prohibitively expensive. If we maintain Laravel-facing stubs or a plugin, this is close to a purpose-built tool.
• If we are a plain Laravel application team with a psalm.xml and a baseline, the fit is weak. Application type errors are already covered by psalm in CI; fixture-level inference assertions solve a problem we don't have.

Migration path

• From nothing: start with a handful of .phpt fixtures asserting the inference we care about (e.g. an Eloquent relation accessor resolving to a concrete model type), run via PsalmPhptTestCase with phptDirectory().
• From baseline files (--update-baseline): these are not equivalent and should not be conflated. A baseline suppresses known errors in application code; a fixture suite pins expected analyzer behavior. Baselines tend to grow monotonically and are invisible in review; fixtures are explicit and reviewable. Migrating a handful of high-value baseline entries into fixtures is a legitimate cleanup, not a mechanical port.
• From hand-rolled Psalm invocations + string matching: this is the strongest migration case. Replacing bespoke shell + regex assertions with Result / Outcome / Issue removes parsing code that we would otherwise own forever.

Compatibility

• Composer: Psalm ^6.10 || ^7.0.0-beta16, composer-runtime-api ^2, PHPUnit 10+ attributes. Lock the Psalm version in lockfile and avoid floating to ^7 until we've run the fixture suite against it deliberately.
• phpt semantics: --FILE--, --EXPECT--, --EXPECTF--, --ARGS--, --SKIPIF--, --XFAIL--, --CONFLICTS-- supported; --CLEAN--, --ENV--, --INI-- explicitly rejected with a clear message, as are --EXPECT_EXTERNAL-- / --EXPECTF_EXTERNAL--. Fixtures ported from php-src may need edits.
• Behavioral change to internalize: --ARGS-- appends to configured arguments, and a --config inside it overrides the configured one. A team upgrading from 0.3 with per-test --config will get different behavior than they expect.

Sequencing

1. Feasibility spike (1–2 days). One repo, one Psalm version, 5–10 fixtures. Confirm composer resolution, PHPUnit version, and that PsalmTester::create() + runBatch() behave under our config. Do not touch CI yet.
2. Pilot on the highest-value surface (1 sprint). Pick the inference behavior we've most recently been burned by — a plugin upgrade that changed types silently, or a stub we don't fully trust. Convert that into ~20 fixtures. Measure wall time with and without runBatch() and withConcurrency().
3. Wire to CI as non-blocking. Record psalm-tester: N phpt files (S skipped), G Psalm runs on STDERR as the run signature; keep results advisory long enough to judge flake rate.
4. Harden. Set withTimeout() and withConcurrency() explicitly; rely on per-run isolated cache dirs; ensure a crashed or unexpected-exit Psalm errors rather than passes (this is a fixed behavior in 0.4.0 — verify it in our environment).
5. Gate. Promote to blocking once the suite has run clean for two weeks. Use --XFAIL-- for known upstream Psalm bugs so "expected failure" flips to a failure when fixed — a scheduled reminder, not a permanent skip.
6. Upgrade discipline. Pin the package version; do not float. Treat each upstream minor as a scheduled upgrade pass against UPGRADING.md, the same way we'd treat a framework minor.

Operational Impact

Maintenance

• Low code surface, high version-churn risk. The thing we maintain is fixtures and expectations, not the harness. Expect fixture maintenance to grow roughly linearly with the breadth of inference we choose to assert.
• The 0.4.0 breaking set means upgrade cost is real and must be budgeted, not assumed away: removed progress output (our log-scraping dies), --ARGS-- semantic flip (our configs may silently point elsewhere), withArguments() signature change (our test bootstrap compile-errors — the good case), rejected --EXPECT*_EXTERNAL-- (our fixtures must inline expectations).
• MIT license plus a small, readable codebase means forking is genuinely viable. Budget ~1–2 engineer-weeks if we have to take ownership, and hold that as the contingency rather than a hope.

Support

• Single maintainer, no SLA, 1 star, 0 dependents — there is no community to lean on and no downstream signal that problems get found before we find them. Our support model is: read the source, file an issue, and be prepared to patch or fork.
• Mitigation: pin versions, keep the integration thin (a single TestCase subclass plus PsalmTester::create() calls), and avoid depending on internal helpers like runGroup() documented as "see the source code." Anything we build on internals is ours to maintain.

Scaling

• Batching is the mechanism that makes this viable at framework scale: one Psalm invocation per argument group instead of per fixture, because plugin boot (a full Laravel app) dominates. The published numbers — 758 fixtures from 28.6s to ~10s, and ~4.6s with --filter on one test — are the relevant benchmark and are the strongest single argument for adoption over a naive loop.
• withConcurrency() plus per-run cache directories (--no-cache, isolated per run) bounds resource use and removes cross-run cache contamination. --CONFLICTS-- (php-src semantics) gives conflicting fixtures their own run, prevents overlapping runs sharing a key, and lets all run alone — that is what makes parallelism safe rather than flaky.
• runBatch()'s shared symbol table is the scaling ceiling on the authoring side: names must be globally unique per argument group or you get DuplicateClass / DuplicateFunction. Plan fixture naming conventions up front; retrofitting them across hundreds of fixtures is unpleasant.

Failure modes

Ramp-up

• Engineer-facing learning curve is modest but non-zero: .phpt section semantics, EXPECT vs EXPECTF, SKIPIF returning a reason with the leading skip token stripped, and the batch-mode constraint on global names. A competent PHP engineer is productive in under a day; writing good expectations (exact enough to catch regressions, loose enough not to churn on every Psalm patch) takes a few iterations.
• The organizational ramp-up is the real cost: someone must own the fixture suite, review new fixtures for over-permissive EXPECTF, and action --XFAIL-- flips. Without that owner, this becomes a green checkmark that no longer means anything.
• Minimum viable team footprint: one engineer at ~20% for the pilot and ~5% steady-state, plus a CI configuration review.

Recommendation

Adopt if we maintain Psalm-facing type behavior — a plugin, stub set, or framework integration — and our Psalm version satisfies ^6.10 || ^7.0.0-beta16. The buy-vs-build case is strong: fixture parsing, batched execution, issue diffing, cache isolation, conflict scheduling, and timeout handling are weeks of undifferentiated work, and the Result/Issue API makes downstream gating trivial.

Do not adopt if we are a consumer of Psalm rather than an extender of it. Application type errors are already covered by psalm in CI; a 1-star, 0-dependent, pre-1.0 dev dependency with a beta upstream is not worth the version-conflict and upgrade-burden risk for coverage we already have.

Either way, resolve the metadata question — the listed 2026-09-30 release date — against Packagist before any pinning decision.