Crucible Laravel Package

Technical Evaluation

Context. This updates the v1.1.5 baseline assessment. v1.1.7 is a four-item patch released three days after v1.1.5, and the delta is entirely Pest-parity correction — no new capabilities, no API surface, no dependency changes. Two patch releases in five days on a compatibility layer is a signal in both directions: the maintainer is responsive, and the compatibility surface is still being discovered in production-shaped Laravel suites.

Architecture fit. Unchanged. Crucible remains a require-dev test runner and mutation-testing tool positioned as a Pest-compatible alternative, confined to the test/CI plane with no runtime, database, or queue surface. The substitute-good framing from the baseline still holds: it is a replacement candidate for the Pest dialect layer, the PHPStan bridge, and the mutation engine, not an additive dependency. The v1.1.7 fixes do not shift placement — they sharpen the "what does it replace" question, because three of the four fixes concern Pest's hook model (beforeEach/afterEach/beforeAll/afterAll, including inside describe()), and one concerns Pest.php configuration semantics. Those are exactly the load-bearing behaviors a team inherits when it removes Pest, and they are the parts of the framework least covered by a typical team's own test suite, since they are infrastructure the suite runs on rather than code the suite asserts.

Integration feasibility. Feasible, and marginally more credible than at v1.1.5, but the release reinforces rather than retires the central risk. The fixes are individually correct and each one is a parity bug of the "silent divergence" class rather than a crash class — which is precisely the class a TPM should weight heaviest. Specifically:

• beforeEach/afterEach inside describe() running for every test in the file (including tests declared before the block) is a cross-contamination bug: setup intended for a subset ran for a superset, and teardown ordering could leave state that made unrelated tests pass or fail. This is the single most dangerous shape of bug in a test runner, because it can produce stable false results that a team trusts.
• beforeAll/afterAll inside describe() being accepted and run file-wide is now a collection-time refusal with Pest's message. Correct against Pest, but note the direction of change: a suite that previously ran will now fail before executing a single test. That is a build-breaking behavioral change shipped in a patch release.
• Global afterEach from Pest.php running after the file's own is an ordering divergence with the same contamination profile as the first item, one scope up.
• ->in(__DIR__) matching nothing in Pest.php, silently dropping its class, traits, groups, and hooks, is the most consequential item in the release for a Laravel team. In a stock Laravel Pest setup, uses(TestCase::class)->in(...) is how test suites acquire their base class; a silent no-match there means tests run without the intended bindings, and any trait- or group-dependent behavior disappears without an error. A configuration directive that silently matches nothing is a failure mode with no natural alarm.

The net read: v1.1.7 closes four real divergences, but the pattern over the last two patches is that Crucible's Pest emulation is being validated by external comparison against Pest rather than by a long-lived internal conformance corpus, and the misses cluster in framework plumbing — hooks and configuration — rather than in assertions. The notes cite a "Hallmark cross-check" for the describe() fix alongside the previously referenced Pest-oracle compat-check. That a second named cross-check appears is a positive assurance signal, but its relationship to compat-check is not stated in the notes and a TPM should not assume the two are equivalent in coverage or cadence.

Technical risk.

• Silent divergence remains the dominant risk, and this release shows where. v1.1.5 was assertion semantics; v1.1.7 is hook scoping and configuration resolution. The uncovered surface is not shrinking in a predictable direction, which means the residual-risk estimate should not be revised downward just because four bugs closed.
• Hook-order parity is version-coupled and hard to test locally. A team cannot easily prove its Crucible hook ordering matches Pest's across Pest.php globals, file-level hooks, and nested describe() blocks. compat-check is the only practical oracle, and it must be run against suites that actually exercise those constructs — a suite with no describe() blocks and no Pest.php bindings will go green and prove very little.
• Patch releases can break collection. The beforeAll-in-describe() refusal converts a passing suite into a collection failure on upgrade. No deprecation window was offered, which makes the baseline's BC-policy question more pointed, not less.
• Configuration-form coverage is narrow. The fix is specified for absolute paths (->in(__DIR__)). Relative paths, globs, arrays, and chained ->in() calls are not addressed in the notes and should be assumed unvalidated until proven otherwise.
• Compat is version-coupled. Unchanged from baseline: behavior is defined relative to Pest and pest-plugin-* versions, and a Pest minor bump remains a live risk to the suite.
• Mutation verdict invalidation. The v1.1.5 stale-verdict fix and its one-time full re-judge cost spike are still in front of any team that adopted at v1.1.5 and has since resumed crucible mutate.
• Sharding is not duration-balanced. Unchanged; do not assume linear speedup.

Key questions. (1) Is Crucible intended as a full Pest replacement or a coexisting mutation/coverage layer — and does the vendor support the former? (2) What is the Pest version-support policy and the tested PHP/Laravel matrix? (3) Licensing and commercial support terms? (4) What is the deprecation/BC policy, given that patch releases now carry both behavioral changes and a new hard failure (collection refusal), with the Quirk::ThrowInterfaceAsMessage flag as the only demonstrated escape hatch? (5) How long must suites stay green under compat-check before Pest can be removed? (6) New: what is Hallmark, how does it relate to compat-check, and does it cover hooks and Pest.php resolution as well as it covers the expect() dialect? (7) New: which Pest.php path forms (absolute, relative, glob, array, chained) are considered supported, and how many of them are covered by the vendor's own cross-check?

Integration Approach

Stack fit. Unchanged: stock Laravel + Pest reference target, modern PHP 8.x matrix, a coverage driver (pcov via 1.0.12 in all mode, Xdebug assumed), JUnit XML enabled for the oracle path. No application-code changes. One addition for the pilot: the target repository must actually exercise the constructs this release touched — at least one describe() block with its own hooks, a Pest.php with ->in(...) bindings, and a global afterEach. A pilot on a suite without those proves nothing about v1.1.7.

Migration path. Sequence from the baseline is unchanged; steps 1–5 stand. Insert two gates:

1. Pin Crucible at v1.1.7 (or later) as require-dev alongside the existing Pest stack in a single pilot repository — do not adopt v1.1.5 or v1.1.6 now that 1.1.7 carries hook-semantics corrections.
2. Run compat-check non-blocking in CI with Pest as oracle; ensure the pilot suite exercises describe() hooks, global Pest.php hooks, and ->in() bindings so the diff has something to bite on.
3. Audit Pest.php explicitly before migrating. For each ->in(...) call, record the current path form and confirm under Crucible that the intended class, traits, groups, and hooks actually bind. Do this as a manual check with a deliberately failing expectation, not by observing green tests — a silent no-match is green.
4. Audit hook ordering by observation, not assertion. Add a small set of sentinel tests that record setup/teardown order for file-level hooks, describe()-nested hooks, and the global afterEach. Keep them in the pilot. They are the regression test for this entire bug class.
5. Escalate compat-check to blocking once the pilot shows zero drift across a stable window and a Pest minor bump.
6. Migrate the dialect incrementally using the manual's "where the Pest dialect differs from Pest" section, then retire Pest only after crucible mutate has run clean for a full release cycle.

Compatibility. Pin Crucible, Pest, pest-plugin-laravel, pest-plugin-phpstan, and the coverage driver together; any one moving triggers a compat-check re-run. The Quirk::ThrowInterfaceAsMessage flag remains a documented, explicit per-suite decision. Add a standing rule from this release: treat every Crucible patch as potentially collection-breaking, and upgrade it in a PR that runs the full suite rather than as a background dependency bump.

Sequencing. Read-only surfaces first (execution + compat-check), then hook and Pest.php parity — promoted ahead of the general dialect work from the baseline, because this release demonstrates that plumbing parity is where the vendor is still finding bugs and where failures are quiet — then assertions/dialect, then PHPStan integration, then coverage/sharding, and mutation testing last.

Operational Impact

Maintenance. Unchanged in shape, higher in frequency than the baseline implied. Two patches in five days means dependency-bump discipline matters: schedule Crucible upgrades, review them against compat-check output, and do not let them ride in with unrelated changes. The hook-sentinel tests proposed above are the durable artifact that reduces this cost over time.

Support. The vendor's cross-check discipline remains the primary assurance, and the addition of a named second check (Hallmark) is a mild positive. Still no evidence of LTS branches or a support SLA. For a dev-only dependency that is tolerable, but it continues to argue against making Crucible a required release gate before a pilot has accumulated history across at least one Pest bump.

Scaling. Unchanged. Sharding works but is not duration-balanced — size shards empirically. Mutation testing scales with test count and coverage fidelity; keep coverage fresh for impact selection.

Failure modes to monitor.

• Silent false greens from assertion-semantics drift — unchanged; keep compat-check in the loop rather than trusting a green Crucible run alone.
• New: silently dropped Pest.php bindings. Though the absolute-path case is fixed, any unmatched ->in(...) form would drop class, traits, groups, and hooks with no error. Alert on tests running with an unexpected base class or without expected traits/groups.
• New: cross-scope hook leakage. describe() hooks escaping their block, or a global afterEach running in the wrong order relative to file hooks, shows up as order-dependent flakes or, worse, consistent passes caused by leftover state. The sentinel tests are the mitigation.
• New: collection-time refusal. After upgrade, a suite using beforeAll/afterAll inside describe() will be refused before running. Fail the build loudly and treat it as a known parity correction, not a Crucible defect — but budget the migration work.
• Exit 255 / no-tests-run. Improved diagnostics now distinguish "Crucible errored" from "PHPUnit oracle" — alert on the former.
• Signal-killed coverage workers. Reported with signal and driver now; treat as hard CI failure.
• Missing JUnit file from the oracle. Validate artifacts exist before diffing.
• Stale mutation verdicts after upgrade. The one-time full re-judge spikes CI time; schedule the first post-upgrade mutate off the critical path.
• PHPStan $this->user false positive. Fixed in v1.1.5, but any static-analysis regression should be triaged as a parity bug against pest-plugin-phpstan.

Ramp-up. Low for engineers fluent in Pest — same vocabulary and Laravel test ergonomics. The learning load shifts slightly toward the test harness itself: engineers need to understand how Crucible resolves Pest.php bindings and orders hooks across file, block, and global scopes, because those are now demonstrated areas of divergence. Add to the internal "where this differs from Pest" note a short section on hook scoping and a required manual verification step for ->in() bindings. Estimate one engineer-week for the pilot plus the compat-check stabilization window; add roughly a day for the Pest.php and hook-order audit on a mid-sized suite.