Phpunit Slow Test Detector Laravel Package

Technical Evaluation

What it is (and isn't): ergebnis/phpunit-slow-test-detector remains a dev-only PHPUnit extension that measures per-test wall-clock duration and reports tests exceeding a configured threshold. As of 2.27.0 it is no longer only a console table: the release adds GitHub Actions workflow-command annotations for slow tests (PR #775) and refactors report emission behind an extracted Printer abstraction (PR #785). That is a material change to the integration story, not cosmetic. It still ships no runtime code, no service provider, no container bindings, and stays entirely in require-dev plus a few lines of phpunit.xml.

Architecture fit: Still excellent. Same lifecycle seam (extensions for PHPUnit 10–13), no coupling to application code, no deployable-artifact cost. Two new wrinkles: (a) the extension now writes to a second channel — the CI provider's annotation API via stdout workflow commands — so it has a hosting-environment dependency it did not have before; (b) the Printer extraction creates an internal seam that likely admits alternative output formats later, which is architecturally welcome but is currently an internal refactor, not a documented extension point. Do not assume a supported custom-printer plugin API until the docs say so.

Integration feasibility: Still high, and now meaningfully higher-value. Effort estimate shifts slightly: under an hour for a local console-only run (unchanged), plus roughly a half-day to validate annotation behavior on a scratch PR (GitHub Actions only), on top of the prior one engineering day for threshold tuning. Teams on GitLab CI, Jenkins, or Buildkite get no benefit from the new feature and are back to log-scraping — that asymmetry is worth stating plainly in the ADR so nobody assumes parity across pipelines.

Technical risk: Net reduction in adoption risk, small increase in versioning risk.

• The largest prior objection — "the build stays green and nobody reads the report" — is substantially weakened. Annotations surface slow tests in the PR diff/checks view, which is the one place engineers reliably look. This is the single most consequential change in the release.
• Still not a gate. Annotations are warning-level signal; PHPUnit's native enforce-time-limit remains the only actual failing mechanism. Teams wanting enforcement still build it themselves.
• New failure mode: annotation volume. GitHub caps annotations per step/job, so a large suite with many slow tests may silently drop annotations. The cap interacts badly with sharded runs, each of which annotates independently.
• Versioning discipline softened: 2.27.0 is a minor bump that drops PHP 7.0–7.3 support. Installability-only breakage, but it means "minor bump" can no longer be treated as strictly non-breaking in an auto-merge flow.
• Bus factor unchanged: single maintainer, small-star OSS. Mitigating signal: the project now auto-merges its own Dependabot PRs and carries heavy release automation, which implies real confidence in its own test coverage — but it also means release cadence is governed by one person's automation policy.

Key questions for the vendor/owner:

1. How are GitHub Actions annotations enabled — auto-detected from GITHUB_ACTIONS, opt-in config, or unconditional when the env var is present? Does it stay quiet outside GitHub Actions, or emit stray ::warning:: lines into other CI logs?
2. What happens when the annotation count exceeds GitHub's per-job cap — silent truncation, or an aggregated summary annotation for the remainder?
3. Does each parallel/sharded run annotate independently, and is the annotation set deduplicated or aggregated across shards?
4. Is the extracted Printer a documented, configurable extension point (custom printer class in phpunit.xml), and does that open a supported machine-readable (JSON) format? This is the answer to the long-standing "dashboards require parsing" gap.
5. Does the PHP-support drop policy (7.0–7.3 removed in a minor) follow a published deprecation notice, and is PHP-version support treated as explicitly non-BC?
6. Previously open and still open: behavior under php artisan test --parallel/paratest, attribution of setUp/tearDown and data-provider time, Pest v2/v3 compatibility, stderr redirection for log consumers.
7. Build-vs-buy still stands, but the calculus improved: computing top-N from --log-junit no longer matches the annotation UX without extra plumbing, so the buy case is stronger than in the prior review.

Integration Approach

Stack fit: Composer-native and Laravel-idiomatic, unchanged. Laravel 10/11/12 ship PHPUnit 10/11/12 with the extensions schema; the snippet drops straight into the root phpunit.xml. Prefer the Composer package over the PHAR (the release includes PHAR-build fixes, but that path remains irrelevant for Composer-managed apps). The new annotation feature is GitHub Actions-specific — treat it as an additive capability, not a replacement for the console report.

Migration path: Still purely additive and reversible, same five-step ratchet, with one insertion:

1. Local opt-in on one developer machine (defaults: 500 ms, top 10) — console output only.
2. Non-blocking CI job on the main branch, output to the build log.
3. New: a scratch PR to validate annotation emission — confirm volume, confirm non-GitHub jobs are unaffected, confirm the annotation cap behaves as expected.
4. Baseline capture, then per-suite maximum-duration tuning (unit 100–200 ms; feature/http 1000 ms+).
5. Raise maximum-count for large suites, or accept truncation.
6. Optional enforcement against a checked-in baseline — keep out of the first PR. If annotation noise proves high, tune before enabling annotations repo-wide rather than disabling the extension.

Compatibility: Confirm the matrix explicitly. The 2.27.0 support-floor drop (PHP 7.0–7.3 removed) affects a narrow edge: Laravel 8 apps still on PHP 7.3. Everyone on PHP 7.4+ and PHPUnit 9.5+ is unaffected in principle, but pin ^2.27 only after checking the support matrix, and pin ^2.26 if any package in the monorepo still resolves on PHP ≤7.3. Continue constraining to a caret range with lockfile pinning; never *. PHPUnit-major-to-PHP-major pinning (11→8.2, 12→8.3, 13→8.4) still governs the real upgrade cadence. The legacy listeners path for PHPUnit ≤9 is unchanged and still carries the deprecation tax.

Sequencing: (1) add dependency, (2) register extension in one phpunit.xml, (3) verify parity across local + CI + one parallel run, (4) validate annotations on a scratch PR, (5) tune thresholds per suite, (6) publish a short internal note on reading both the table and the annotations, (7) only then consider gating. Do not bundle with a PHPUnit upgrade — the annotation path is new code and you want a clean bisect if output misbehaves.

Operational Impact

Maintenance: Low but now marginally higher-attention. This stays in the Renovate/Dependabot stream, but 2.27.0 is the proof that a minor bump can drop PHP runtimes — so stop auto-merging patch/minor bumps for this package and route them to the CI/test-infrastructure owner. Expect a handful of bumps per year plus one meaningful bump per PHPUnit major. Ownership unchanged: whoever owns CI/test infrastructure.

Support: Minimal. No on-call surface, no production exposure, no network path at runtime. Worst realistic incidents are "noisy annotations" and "extension class not found," both fixed by deleting a few lines of XML or a config block. The annotated-output feature raises support load slightly in the first weeks — expect a handful of "why is this PR flagged?" questions until thresholds are tuned and the team learns the signal.

Scaling: Compute overhead remains negligible. The scaling caveat is now twofold: sharded runs still emit per-shard top-N (aggregate "top 10" requires concatenating logs), and sharded runs also annotate per-shard, which means the GitHub annotation cap can be hit sooner than expected and the union of annotations across shards may not equal the global worst offenders. For a 5,000-test suite, maximum-count remains the right knob, but it now governs annotation volume as well as table length — tune it with the annotation cap in mind, not just log readability.

Failure modes to plan for:

• Annotation noise / alert fatigue — now the top risk, and worse than before because a low threshold litters every PR's checks view rather than hiding in a log nobody opens. Tune per-suite thresholds before enabling annotations broadly; consider enabling annotations on main-branch runs first, then PRs.
• Silent annotation truncation — GitHub's per-job cap can drop the tail without warning; verify with a deliberately slow test at the bottom of a long list.
• Stray output in non-GitHub CI — if annotation emission is not env-gated, ::warning:: lines pollute GitLab/Jenkins logs and can trip log scrapers. Verify before rollout.
• PHP-unit support drop in a minor — mitigated by pinning, reviewing the matrix before bumping, and not auto-merging this dependency.
• PHPUnit major bump breaks the extension — unchanged, and still the highest-probability breakage; upgrade as a deliberate, isolated change.
• Silent omission — materially reduced by annotations, but not eliminated: annotations are warnings, so a team that learns to ignore yellow checks loses the benefit too.
• Dependency abandonment — unchanged low likelihood. Keep the JUnit-XML parsing fallback in the ADR, though note the gap widened: the fallback replicates the table, not the annotation UX.

Ramp-up: Still essentially zero for engineers. Budget about 30 minutes for a CI owner to wire the extension, an extra 15–30 minutes to validate annotation behavior, and an hour of senior-engineer judgment on per-suite thresholds. The release strengthens the adopt case — PR-visible feedback is what converts a nice-to-have report into something teams actually act on — so this moves from "adopt opportunistically" to "worth a deliberate, time-boxed rollout," with threshold tuning and annotation validation as the two things that must happen before broad enablement.