Black Box Laravel Package

Getting Started

1. Install (dev only)

composer require --dev innmind/black-box

Check composer.json for the required PHP version before you commit — this package tracks recent PHP releases and will not run on older Laravel stacks.

2. Where to look first

• README add example (commutative/associative/identity) — this is the whole mental model: a property is a statement that must hold for every input.
• src/Application.php — the array passed to Application::new([...]) is the only configuration surface. Read that file (or run the binary with --help) before guessing flags.
• src/Set/ — the catalogue of generators. This is where you will spend 90% of your time.
• src/Runner/Assert.php — the assertion API. It is not PHPUnit's Assert.
• CHANGELOG.md — the API has moved between majors (Application::run → tryToProve); pin a version and read the changelog on upgrade.

3. First use case

Pick a pure piece of domain logic — a value object, a slugger, a money/rounding helper, a date-range normaliser. Do not start with a controller test.

Create tests/Property/ (outside the folder PHPUnit scans) with one entry file per domain area:

<?php // tests/Property/domain.php
declare(strict_types=1);

use Innmind\BlackBox\{Application, Set, Runner\Assert, Prove};

require __DIR__.'/../../vendor/autoload.php';

Application::new([])->tryToProve(static function(Prove $prove) {
    yield $prove
        ->proof('Money::add is commutative')
        ->given(Set::integers(), Set::integers())
        ->test(static fn(Assert $assert, int $a, int $b) => $assert->same(
            Money::of($a)->add(Money::of($b)),
            Money::of($b)->add(Money::of($a)),
        ));
});

4. Run it

vendor/bin/blackbox tests/Property

Add a composer script so it is not forgotten:

"scripts": {
    "test": ["@php artisan test", "blackbox tests/Property"]
}

It does not hook into php artisan test. Treat it as a second, complementary suite in CI.

Implementation Patterns

The four property shapes worth memorising

Build domain objects by mapping primitives, not by generating them

$emails = Set::strings()->map(static fn(string $s) => Email::of($s));

Generate the primitive and map it. Shrinking then operates on the simple value, which is what makes failure reports readable. If your constructor throws, wrap the map so it produces a valid value instead — a generator that throws mid-shrink is the classic way to make a PBT suite useless.

Compose sets for realistic inputs

Set::integers(), Set::strings(), Set::sequenceOf(...), Set::either(...), Set::elements(...), plus range/length narrowing on the primitive sets. Check src/Set/ for the exact list in your pinned version — it grows between releases.

Ports-and-adapters is the integration trick with Laravel

Proofs run many times per case (default order of ~100 scenarios). Anything touching MySQL, Redis, HTTP, or the filesystem multiplies that cost by ~100. So: generate plain values in the set, resolve an in-memory / fake implementation of your interface in the proof. If you genuinely need the framework container, bootstrap it once per process:

$app = require __DIR__.'/../../bootstrap/app.php';
$app->make(Illuminate\Contracts\Console\Kernel::class)->bootstrap();

Then bind fakes into the container before the tryToProve call.

Split by intent, not by class

One entry file per bounded context (domain.php, money.php, parsing.php). Use ->proof('...') names as stable identifiers — they are your filter handles and your CI failure labels.

Replay failures deterministically

When a proof fails, the runner reports the generated values/seed. Capture the reported seed in the failure's issue ticket and re-run with it to avoid re-rolling for the same counterexample. Always reproduce a failure before "fixing" it.

Keep PHPUnit for what it is good at

HTTP tests, middleware, auth, RefreshDatabase, queues, Http::fake(), Dusk — all PHPUnit/Pest. BlackBox covers the pure kernel underneath. This split is the whole day-to-day value: PHPUnit proves examples work, BlackBox proves the rule holds.

Gotchas and Tips

• It is not a PHPUnit plugin. No $this->assertSame, no RefreshDatabase, no Laravel test-case lifecycle, no assertDatabaseHas. Every attempt to port a feature test into a proof ends in a slow, flaky mess.
• Namespace collision: Innmind\BlackBox\Runner\Assert vs PHPUnit\Framework\Assert, and Set vs Illuminate\Support\Set-ish helpers. Always alias on import in files that mix ecosystems.
• Keep proof files out of PHPUnit's scan path. PHPUnit collects *Test.php. A proof file that requires and executes Application::new(...) at top level will be executed as a side effect if it is ever loaded by PHPUnit. Put it in tests/Property/ with a name that doesn't match your suite suffix, and point phpunit.xml <testsuite> at tests/Unit + tests/Feature only.
• Assertions throw and abort. The first failure ends the scenario so the runner can shrink. You cannot collect multiple assertion failures in one pass — order assertions so the most specific check runs first, or split into separate ->proof() entries.
• Shrinking quality depends entirely on your set. Set::map with a lossy or non-injective function, filter that rejects most values, or recursive sets without a size bound all produce either poor counterexamples or non-terminating runs. Bound sizes explicitly.
• Set::filter is a trap under time pressure. It silently retries and can starve or loop. Prefer generating values that are valid by construction.
• Failure messages may not print your input. If the runner output doesn't show the offending value, embed context in the assertion itself: $assert->same(['input' => $a, 'result' => $out], ['input' => $a, 'result' => $expected]). Ugly but debuggable.
• Side effects replay. No real DB writes, no mail, no HTTP, no now()-dependent values, no static caches that accumulate across scenarios (Eloquent model booting and container singletons are the usual leaks). If a proof must use a transaction, remember setup runs per scenario.
• Config quirks live in the Application::new([...]) array. Undocumented-looking keys are common; read src/Application.php rather than trial-and-error. The same applies to CLI flags — --help is authoritative for the installed version.
• Extension point: implement Innmind\BlackBox\Set (or use the provided compose/map combinators) to ship a reusable generator for a domain concept — e.g. Set::money(), Set::sku(). Build these once per project; they pay back immediately and are the main reason to choose this over PHPUnit data providers.
• Maintenance risk: 47 stars, effectively a single author, zero dependents, fast-moving API. Isolate it to tests/Property/ so a breaking major only ever touches proof files, pin to a tight constraint, and be prepared to port your generators to another PBT library if it stalls. The properties you write are the durable asset; the API around them is not.