Benchmark Laravel Package

Getting Started

1. Install as a dev-only dependency

composer require dragon-code/benchmark --dev

The package brings a global bench() helper and a Benchmark class into your autoload graph. Because it's --dev, Composer will exclude it from composer install --no-dev on production — which is what you want, since it's purely a development/CI tool.

2. Where to look first

Read the README in this order, because each section builds on the previous one:

1. Usage — the fluent API shape: bench()->compare(...)->toConsole().
2. Iterations Count — the default is 100 iterations per callback, and this is the biggest performance/cost knob you control.
3. Warm-up — disabled by default, and it's the single change that most improves numeric stability for anything touching Laravel's container, autoloading, or the DB.
4. Results — toConsole() for eyeballing, toData() for programmatic use, toAssert() for tests.
5. Regression Testing — the feature that turns this from a curiosity into something you can wire into CI.

3. Your first real benchmark

Don't start with a trivial strlen() vs count() comparison. Start with an actual decision you're facing in your codebase. A classic Laravel-shaped first use case: "Is Collection::map() + filter() meaningfully slower than a single foreach for the sizes we actually deal with?"

use function DragonCode\Benchmark\bench;

bench()
    ->iterations(50)
    ->warmup(3)
    ->compare(
        'collection chain' => function () {
            return collect(range(1, 1000))
                ->map(fn ($i) => $i * 2)
                ->filter(fn ($i) => $i % 3 === 0)
                ->values()
                ->all();
        },
        'plain foreach' => function () {
            $out = [];
            foreach (range(1, 1000) as $i) {
                $v = $i * 2;
                if ($v % 3 === 0) {
                    $out[] = $v;
                }
            }
            return $out;
        },
    )
    ->toConsole();

Run it from php artisan tinker, a scratch file under tests/, or a dedicated command. The output is a table with min / max / avg / total per callback, plus an order row ranking them by average — order: 1 is the fastest. Read the order row first; the millisecond columns are there to tell you how much it matters, not whether it matters.

The immediate payoff is not micro-optimisation. It's discovering that the "obviously slower" version is 4% slower and therefore not worth refactoring — or that it's 8× slower, which justifies the rewrite.

Implementation Patterns

Pattern 1: Named closures as a self-documenting table

Pass callbacks as named arguments so the option label appears as the column header. This makes the output readable when you paste it into a PR description.

bench()->compare(
    'eager load' => fn () => User::with('posts')->take(100)->get(),
    'lazy load'  => fn () => User::take(100)->get()->each->posts,
)->toConsole();

Associative-array form (compare(['eager' => fn() => ..., 'lazy' => fn() => ...])) is the equivalent — useful when you're generating the option list dynamically. Unnamed positional callbacks work too and get numeric labels, but you lose all readability.

Pattern 2: Benchmark from a Pest/PHPUnit test with assertions

The highest-value pattern for a Laravel team is to make benchmarks assertions in your test suite, so a slow path fails CI instead of relying on someone remembering to run it.

use DragonCode\Benchmark\Benchmark;

it('resolves config-backed values in under 1ms', function () {
    $results = new Benchmark()
        ->iterations(20)
        ->warmup(2)
        ->compare(
            'config repository' => fn () => config('app.name'),
            'env helper'        => fn () => env('APP_NAME'),
        )
        ->toData();

    expect($results)->toHaveKeys(['config repository', 'env helper']);
});

And with toAssert() for range checks — note that from and till are both optional, so you can assert an upper bound only:

new Benchmark()
    ->iterations(20)
    ->warmup(2)
    ->compare(foo: fn () => $this->service->resolve())
    ->toAssert()
    ->toBeAvgTime(till: 3)          // must average under 3 ms
    ->toBeDeviationTime(from: -0.5, till: 0.5);

Use toBeAvgTime / toBeMaxTime for ceilings, and reserve toBeDeviationTime for flagging noisy benchmarks — a wide deviation means your measurement is unreliable, which is a signal to add warm-up or reduce external interference, not a signal about the code.

Pattern 3: Regression baselines in CI

This is the pattern that earns the package its place in a repo. toBeRegressionTime() / toBeRegressionMemory() compare the current run against a .snap file, and throw AssertionError when the regression exceeds the threshold.

new Benchmark()
    ->iterations(50)
    ->warmup(3)
    ->snapshots(directory: __DIR__ . '/.benchmarks')
    ->compare(
        'report builder' => fn () => $this->reports->build(),
    )
    ->toAssert()
    ->toBeRegressionTime(max: 20)     // tolerate 20% drift
    ->toBeRegressionMemory(max: 20);

Workflow:

1. First run — no .snap exists, results are written, no assertion. Commit the generated snapshot.
2. Subsequent runs — compared against the baseline; anything above max percent fails.
3. Deliberate improvement or refactor — delete the specific .snap file, rerun, commit the new baseline. The README calls this out explicitly: deleting the snapshots is the documented reset mechanism.

Set max generously (15–25%) unless you run benchmarks on a dedicated, quiet runner. CI containers have noisy neighbours; a 5% threshold will produce flaky red builds that everyone learns to ignore — the worst outcome for a benchmark suite.

Pattern 4: beforeEach for realistic data, warmup for realistic timing

The classic mistake is benchmarking empty tables or cold caches. Wire up fixture data once, reset it per iteration:

new Benchmark()
    ->iterations(30)
    ->warmup(3)
    ->beforeEach(fn (int|string $name, int $iteration) => [
        'user' => User::factory()->make(['id' => $iteration + 1]),
    ])
    ->compare(
        'accessor' => fn (array $before) => $before['user']->display_name,
        'raw attr' => fn (array $before) => $before['user']->name,
    )
    ->toConsole();

The return value of beforeEach is injected as the first argument of the compare callback. This is the package's only data-plumbing mechanism, and it keeps each iteration independent. beforeEach / afterEach also fire during warm-up runs, so your setup stays consistent — only the warm-up timings are discarded.

Use before() / after() (fired once per callback, receiving the callback name) for expensive one-time setup, such as booting a container binding or seeding a real table.

Pattern 5: toData() for custom reporting

toConsole() is ASCII art; toData() is an array of DragonCode\Benchmark\Data\ResultData DTOs keyed by callback name, each with min, max, avg, total as MetricData (time, memory) and, when deviations() is enabled, a DeviationData containing percent. Reach for this when you want to:

• Emit a Markdown table in a PR comment from a GitHub Action.
• Push numbers to a dashboard or a storage/benchmarks.json artifact.
• Do your own ranking/threshold logic that toAssert() doesn't cover.

$data = new Benchmark()->deviations()->compare(...)->toData();

foreach ($data as $name => $result) {
    $this->line(sprintf('%s: avg %.4f', $name, $result->avg->time));
}

Pattern 6: Comparing query strategies with real DB work

Laravel benchmarks that hit the database are the ones people actually want. Keep iterations low (5–15) because each iteration is a real round trip, and always warm up so the first query doesn't pay connection-setup cost:

new Benchmark()
    ->iterations(10)
    ->warmup(2)
    ->compare(
        'query builder' => fn () => DB::table('orders')->where('status', 'paid')->count(),
        'eloquent'      => fn () => Order::where('status', 'paid')->count(),
        'raw sql'       => fn () => DB::selectOne("select count(*) from orders where status = 'paid'"),
    )
    ->toConsole();

Wrap it in a transaction (DB::beginTransaction() … DB::rollBack()) or a RefreshDatabase-style trait so repeated iterations against a mutating dataset don't skew the numbers.

Gotchas and Tips

The average is trimmed, not raw

"When more than 9 iterations are used, the top and bottom 10% of results are excluded from the average calculation."

This is a good default — it prevents a single GC pause or disk hiccup from dominating avg — but it means avg is not total / iterations. If you're validating the output against your own arithmetic, or comparing against numbers from a different tool, expect a mismatch and don't file it as a bug. min and max are reported from the untrimmed set, so max can look alarmingly large on a run with one outlier.

Memory numbers are the weak signal — trust time

Every example in the README shows 0 bytes or a handful of bytes, and the deviation memory rows are all 0. Peak/usage deltas at this granularity are dominated by PHP's allocator behaviour, not by your code. Use the package for time, and use Blackfire/Telescope/Xdebug for memory. If a toBeMaxMemory(till: 1024) assertion is flaky, that's expected — loosen it or drop it.

Snapshot paths are derived from call site, not callback name

Snapshots are "stored per call site (subdirectory derived from the caller file and line)." Consequences:

• Moving the benchmark block down ten lines silently creates a new baseline. You'll lose the regression check without any error.
• Refactoring a benchmark into a helper method changes the call site and orphans the old snapshot.
• Adding a new callback to an existing compare() may or may not be covered depending on how the snapshot is keyed — verify by running twice after any edit.

Mitigation: always pass an explicit ->snapshots(directory: ...) so the root is predictable, and commit .benchmarks/*.snap to VCS as the README advises. Review snapshot-file diffs in PRs the same way you review lockfile changes — a suspicious jump in the baseline is a warning sign that someone re-ran the benchmark on a loaded laptop.

Warm-up is off by default and you almost always want it

The README added warm-up in 4.6.0 and fixed its iteration handling in 4.6.1. If you're on 4.6.0, upgrade — the warm-up logic had a bug. For anything involving Laravel's container, Eloquent model booting, or the first autoload of a class, run ->warmup(2) or ->warmup(3). Without it, the first iteration carries cold-cache cost that no amount of trimming fully removes. Warm-up iterations execute beforeEach / afterEach and advance the progress bar but are excluded from statistics — so they look like real iterations in the progress output. Don't be confused by the extra ticks.

deviations() multiplies your runtime

deviations(4) repeats all loops four times. Combined with the default 100 iterations and several callbacks, a run that took 10 seconds now takes 40+. For deviation analysis, drop iterations() to 10–20 — the whole point of deviation measurement is to see run-to-run variance, and you get that from repeated batches, not from a huge batch count.

toAssert() is a separate chain — don't try to keep calling toConsole()

The documented flow is compare(...)->toAssert()->toBeAvgTime(...). toAssert() transitions you into an assertion object with its own methods. If you want both console output and assertions, benchmark once, call toConsole(), and either call toAssert() in a separate run or assert against toData() yourself with Pest's expect(). Chaining toConsole()->toAssert() is not a documented contract.

Assertion units are milliseconds, not seconds

Console output is in ms (14.3472 ms), and the toAssert() examples use values like till: 3 described as "between 0.5 and 3 ms." But the toData() example dump shows total: 0.7747 for what the console renders as ~774 ms — i.e. the DTO appears to carry raw seconds. Before writing assertions, run the benchmark once and print toData() to confirm the unit on your installed version. A 1000× unit mistake turns a meaningful ceiling into a no-op or an instant failure.

Disable the progress bar in CI and in tests

Progress bars write escape sequences to stdout; in PHPUnit/Pest they can clutter output, and in CI logs they produce thousands of ANSI lines. Add ->disableProgressBar() in any benchmark that runs under test or in a pipeline.

Keep iterations low for anything touching I/O

100 iterations × 4 callbacks × a 50 ms query = 20 seconds per run, every run. Rule of thumb:

Beware closures that capture and mutate shared state

A benchmark closure that appends to an outer array, or mutates a shared model instance, will produce progressively slower iterations and a garbage max. Keep every callback pure with respect to shared state — allocate inside the closure. This is another reason to lean on beforeEach's return value rather than top-level variables.

Don't nest benchmarks

Calling bench() or new Benchmark() inside a compare callback will interleave progress bars, iterations, and timers. If you need to compare nested scenarios, benchmark each level in a separate script run.

The bench() helper is global

The package registers a global bench() function. If you already have a bench() helper (some profiling or testing setups do), you'll get a redeclaration fatal. The class API (new Benchmark() or Benchmark::make()) is the escape hatch — and honestly the class form is better in tests anyway, since it's mockable in intent and greppable.

121 stars, single-maintainer, actively released — use it accordingly

This is a small, focused utility from a prolific package author (DragonCode / Andrey Helldar). It works, it's MIT, and it's a --dev dependency — so the blast radius is limited. But with effectively zero dependents and a bus factor of one, don't build critical CI gates on the snapshot feature alone. Practical stance: use it freely for exploration and for non-blocking "report the numbers" jobs; for a hard release-blocking performance gate, either keep thresholds loose or pair it with a system you control end-to-end.

Quick comparison against the alternatives

• phpbench/phpbench — heavier, config-file driven, statistically rigorous (multiple revolutions, standard deviation in the report). Choose it if you need publishable numbers or cross-machine comparisons.
• Laravel's own Benchmark helper (Illuminate\Support\Benchmark) — already in the framework, returns plain arrays, no table output or snapshots. Fine for a one-liner; you're here because you want the table, the beforeEach plumbing, and the snapshot regression check.
• dragon-code/benchmark — best-in-class when the developer experience matters: you want a readable table in ten seconds, or a one-line regression guard in a Pest test.

That last point is the honest assessment. The package's value is low friction, not statistical rigour. Reach for it when the question is "is option A meaningfully slower than option B in our app, on this data?" — and accept that the answer is directional, not peer-reviewed.