Pao Laravel Package

Technical Evaluation

Architecture fit

PAO is a developer-loop output shim, not a runtime dependency. It installs via composer require laravel/pao --dev, hooks into PHPUnit, Pest, Paratest, PHPStan, Rector and Laravel Artisan through Composer's autoloader, and rewrites their console output into compact JSON only when it detects an AI agent in the environment. It touches no application code, no HTTP surface, no database, and no queue path — so it carries no production reliability or performance exposure whatsoever. That makes it a zero-blast-radius decision from an architecture standpoint.

The strategic fit is conditional and specific: PAO's entire value is a function of how much of our PHP engineering work is done by agents (Claude Code, Cursor, Devin, Gemini CLI). Where that holds, the payoff is direct — the README's 1,002-test example collapses from hundreds of lines of progress dots to ~8 JSON fields, and Artisan commands like about, db:show and migrate:status drop 70–75% in tokens. Fewer tokens per tool call means more of the agent's context window spent on the actual code, which is the lever a TPM can actually point at: lower inference spend and higher agent task completion on the same repos.

Where agents are not in use, PAO is inert — it changes nothing and delivers nothing. That asymmetry is the reason to run a bounded pilot rather than a platform-wide mandate.

Integration feasibility

Integration friction is genuinely low: one dev dependency, no config file, no CI change, Laravel 12+ service provider auto-discovery. Reversibility is a single line removal from composer.json plus a lockfile update.

The hard gates must be checked before committing engineering time:

• PHP 8.3+ — any repo on 8.2 is out of scope.
• PHPUnit 12–13, Pest 4–5, Paratest, PHPStan, Rector, Laravel 12+ — repos on PHPUnit 11 or older Pest cannot adopt.
• Dev-only dependency still participates in Composer resolution. On long-lived or dependency-constrained applications, transitive constraint conflicts are the most likely practical blocker; pilot on modern, greenfield-adjacent repos first.

Activation is the design property that makes this sellable internally: because human terminal output stays byte-identical, existing CI log consumers, log-scraping dashboards, and developer muscle memory are unaffected. That removes the usual objection to output-format changes and lets us phase adoption per repo rather than as a flag day.

Technical risk

• Agent detection is a heuristic. It keys off environment signals from fast-moving third-party tools. False negatives degrade gracefully into normal output. False positives are the real hazard: JSON emitted where a human or a log parser expects text, most plausibly inside CI containers that set similar environment variables. We need to know whether agent-detection env vars leak into our CI images.
• Exit-code correctness is the gating risk. The changelog shows a repeated pattern of exit-code and runtime-exit fixes (exit code 1 on phpunit, test results being displayed on runtime exits, bypass formatter for raw output). If PAO ever reports success on a failing suite, we have silently disabled our quality gate — a far worse outcome than verbose logs. This must be explicitly verified across pass, fail, error, timeout and partial-run scenarios before it goes anywhere near a repo that gates merges.
• Upstream coupling is structural. Most releases are compatibility patches for PHPUnit/Pest/Paratest printer internals (PHPUnit 13.3, Paratest 13.2.x, Pest fatal errors, beforeEach/afterEach failures). Adoption means upgrading PAO in lockstep with the test toolchain indefinitely — a recurring maintenance tax, not a one-time install.
• Deliberate truncation. PHPStan output is grouped by file and capped at 30 errors. That's a sound token decision but an inherited judgment call; on large legacy codebases an agent may never see the full error set, which can produce incomplete remediation.
• Thin maturity signals. 728 stars but 0 dependents, documentation limited to README plus changelog, no published JSON schema, and a namespace migration from nunomaduro/pao to laravel/pao at v1.0.6. We should pin the canonical package name in internal docs and treat the output shape as unstable across minor versions.

Key questions

1. What share of our PHP engineers use an agent daily, and on which repos? This single number determines whether the pilot is worth running at all.
2. Do our largest suites exceed a few hundred tests — the threshold where context crowding actually degrades agent behavior?
3. Does PAO's exit code match native tool exit codes on failure, error, timeout, and partial/interrupted runs?
4. Do our CI runners and container images carry any of the environment signals PAO uses for agent detection?
5. What is the token spend baseline on agent-driven test/analysis calls today, so we can attribute savings rather than assume them?
6. Who owns the lockstep upgrade obligation when our test toolchain bumps?

Integration Approach

Stack fit

The best-fit profile is a Laravel 12+ repository on PHP 8.3+, Pest 4–5 or PHPUnit 12–13, with PHPStan and Rector already in the dev toolchain, and a team that routinely drives the terminal through an agent. That repo gets the full benefit set: structured test JSON, structured static-analysis JSON, native Rector JSON, and cleaned Artisan output — all behind a single --dev install.

Poor-fit profiles: PHP 8.2-or-earlier applications, PHPUnit 11 or older Pest lines, monorepos with heavily pinned dependency graphs, and teams with no agent usage. These should be explicitly excluded from the pilot rather than attempted, because the failure mode there is Composer resolution friction with no offsetting benefit.

A secondary fit worth naming: PAO is tool-agnostic across frameworks (Symfony, Laminas, vanilla PHP). If we have non-Laravel PHP repos, the test/analysis half of the value still applies even though the Artisan formatter does not — a useful way to widen the pilot without widening the risk.

Migration path

Treat this as an opt-in, per-repo experiment rather than a platform standard:

1. Baseline. Pick two or three candidate repos and capture current agent token consumption and task success on test/analysis loops. Without a baseline, the 99.8% claim is unverifiable on our own work.
2. Correctness spike. Before any team-wide exposure, run the exit-code matrix (pass, fail, error, timeout, interrupted) on a pilot repo and confirm PAO's exit codes match native. This is the go/no-go gate.
3. Shadow exposure. Install on a small number of engineer machines with agent workflows and let it run for two to three weeks. Because human output is unchanged, there is no disruption to measure — only benefit.
4. Evaluate. Compare token spend and agent completion against baseline; collect qualitative feedback on whether agents recover from failures more effectively with structured *_details than with raw terminal noise.
5. Decide. Either standardize via a template composer.json addition for new repos, or remove. Both outcomes are cheap.

Compatibility

Version-pin deliberately. Pin PAO to a specific minor and pair it with a known-good PHPUnit/Pest version; the changelog makes clear that unsupported combinations fail in non-obvious ways (usage with non-supported versions of pest, paratest support with phpunit 13.2.x). Standardize the canonical package name (laravel/pao) in docs and any internal boilerplate, since older references may still point at nunomaduro/pao. Do not rely on the JSON field set being stable across minor versions — if we later build dashboards or agent prompts on top of these fields, that dependency belongs behind a documented contract we control, not on PAO's output directly.

Sequencing

Sequence by value density, not by repo count. First: agent-heavy teams on modern Laravel repos with large Pest suites — highest token reduction, lowest compatibility risk, fastest feedback. Second: repos that also benefit from Artisan cleaning (db:show, migrate:status, about are common agent inspection commands). Third: static-analysis-heavy repos, where PHPStan/Rector JSON has real value but the 30-error cap needs a judgment call from the owning team. Defer everything else until the pilot produces numbers.

Operational Impact

Maintenance

The steady-state cost is the lockstep upgrade obligation with PHPUnit, Pest, Paratest, PHPStan and Rector. Realistically this is a few hours per quarter concentrated on test-toolchain bumps, and it should be assigned to a named owner on the platform or developer-experience team — not left as an orphaned dev dependency. Budget for PAO releases lagging upstream tool releases by days to weeks, and decide in advance whether test-toolchain upgrades wait on PAO compatibility or whether PAO is temporarily removed. Recommendation: remove-first, so toolchain upgrades never block on a dev-convenience package.

Support

Support surface is small and well-bounded. Because human output is unchanged, developer-help requests should be near zero. The realistic support tickets are: "my agent output looks wrong," "Composer won't resolve," and "the JSON shape changed after an upgrade." All three are answerable from the changelog. The genuine gap is the absence of a published output schema and a single-maintainer-heavy contribution history with zero external dependents — there is no community of adopters absorbing edge cases for us. Internal documentation should therefore state plainly what PAO does, which versions are supported, and how to disable it.

Scaling

Scaling characteristics are favorable and unusual: output is constant-size regardless of suite size. A 1,000-test suite and a 10,000-test suite produce roughly the same JSON footprint, so agent context consumption does not grow with test count. That is precisely the property that makes PAO more valuable as repos get larger — the opposite of most tooling, where scale amplifies cost. The two places scaling does not help: PHPStan output caps at 30 errors, so very large legacy codebases get a bounded and possibly incomplete error view; and the Artisan transformation's benefit depends on command verbosity, which varies by command.

Failure modes

• False positive agent detection in CI — JSON lands in a log pipeline expecting human text. Mitigation: verify CI environment variables before rollout; scope installation to developer machines in the pilot.
• Incorrect exit code on failure — the highest-severity failure, since it silently disables the quality gate. Mitigation: explicit exit-code test matrix as a pre-adoption gate, and re-verify on every PAO upgrade.
• Upstream incompatibility after a toolchain bump — PAO silently stops formatting, or worse, interferes. Mitigation: pin versions, run the matrix in CI on upgrade, remove-first policy.
• Truncated static-analysis output — an agent remediates only the visible 30 errors and reports success prematurely. Mitigation: instruct agents to run PHPStan directly for full enumeration on large codebases, or accept iterative passes.
• Agent misses fatal/bootstrap errors — the changelog shows repeated fixes for fatal errors and beforeEach/afterEach failures in Pest. These are the cases where an agent most needs accurate signal; verify explicitly on the pilot repo.

Ramp-up

Ramp-up is effectively zero for engineers: install the package, change nothing else, and human workflows are untouched. The real ramp-up cost is organizational, not technical — baselining agent token spend, defining what "better agent outcomes" means measurably, and securing a named owner for the upgrade obligation. A one-page internal note plus the exit-code verification runbook is sufficient onboarding.

Bottom line for the TPM decision: this is a low-cost, fully reversible experiment with a plausible and measurable upside concentrated in agent-driven PHP workflows. Approve a two-to-three-repo pilot gated on exit-code correctness and a token-spend baseline; do not standardize org-wide until the numbers come back.