Laravel Pennant Experiments Laravel Package

Getting Started

composer require <vendor>/laravel-pennant-experiments
php artisan vendor:publish   # config + migrations
php artisan migrate
php artisan about            # confirms the "Experiments" entry is wired up

First three things to look at, in order:

1. The published config — cookie name/lifetime for guest IDs, read-only default, and any statistics defaults.
2. The gates — viewPennantExperiments and managePennantExperiments are local-only by default. Define them in a service provider before you try the results UI anywhere but local, or you will hit 403s and assume the package is broken.
3. The results command — php artisan pennant-experiments:results is the fastest way to confirm definitions are registered and exposures are flowing.

First use case — a two-variant test on a checkout CTA, defined once in a dedicated provider:

// app/Providers/ExperimentServiceProvider.php  (boot())
Experiments::define('checkout-cta')
    ->variants(['control' => 50, 'urgent' => 50])   // first variant is the control
    ->goals(['purchase']);

Render it, and remember that resolution and exposure are two separate calls:

$variant = Feature::value('checkout-cta');  // assigns (sticky, hashed) — does NOT count
Experiments::expose('checkout-cta');        // records the exposure

In Blade, use @variant (confirm the exact signature in the README) — it records the exposure for you, which is the whole point of using it over Feature::value().

Implementation Patterns

Expose at the render boundary, not in middleware. Exposures should equal "this participant actually saw a variant." Calling experiments()->expose() from a controller that might redirect, or from a base layout shared by 30 pages, inflates the denominator and quietly biases every result. Put @variant around the markup that changes.

Conversions: PHP for domain events, JS for UI intent. Server-side Experiments::convert('checkout-cta', 'purchase') from a listener on your OrderPaid event is the trustworthy path. For things that only exist in the browser (video play, scroll depth, modal open), include @experimentsScript once per layout and either call PennantExperiments.convert(...) or drop data-experiment-convert on the element — the latter is the least invasive option for existing marketing markup.

Let the Login handoff do its job. Guest → user migration is automatic on Login: exposures, conversions and the variant move across, count once, and keep the same variant. The practical rule is don't re-expose after login and don't convert twice in a sign-up flow — the package already deduplicates per scope and goal.

Set the planned sample size before launch. Either supply it directly or set a minimum detectable effect and let the package derive it. This is what makes "Keep running" meaningful. Without it you get the minimum-data rule and an honest shrug; with it, the verdict vocabulary (Better / Worse / No significant difference) actually fires.

Plan the run, then end it in a controlled way. pennant-experiments:end with winner rollout rewrites stored Pennant values so every participant — new and prior — gets the winner, and recording stops. Treat that as a deploy-time action, not a button-press: it changes what production code reads from Feature::value(). "Run again" starts a clean run, which is the correct move when you want to change weights (see gotchas).

Lock the UI down per environment. Read-only mode on in production, managePennantExperiments restricted to an admin list, viewPennantExperiments for whoever reads results. Read-only gives you the dashboard without the ability to end a live test from a browser tab.

Keep eligibility rules narrow and explicit. Excluding staff, internal IPs, and pre-launch accounts at definition time is much cheaper than trying to filter them out of the numbers later — you can't retroactively un-expose someone.

Dashboard integration is optional and decoupled. With amjadiqbal/laravel-pennant-dashboard v1.1+ you get an Experiments tab; on older versions you get a link back. Neither package requires the other, so version them independently.

Test the assignment logic, not the statistics. In Pest/PHPUnit, assert that a given scope resolves to the variant you expect, and flush Pennant's feature cache between tests — stickiness is the feature, and it will leak across test cases otherwise.

Gotchas and Tips

• variant() never counts. Resolving the feature value without expose() (or @variant) produces a results page with traffic but zero exposures. If your N is 0 while conversions exist, this is the bug — conversions are gated on exposure.
• Changing weights mid-test only affects new scopes. The database driver stores the first answer, and hashing is sticky. Reweighting an in-flight test gives you two different splits silently merged into one report. End the run and "Run again" instead.
• Bonferroni will tell you "Keep running" on a real effect. With three variants at α = 0.025, the release's own validation saw p = 0.0006 and still "Keep running" because the planned per-variant N wasn't reached. Read the difference interval and P(beats control) alongside the verdict; don't treat "Keep running" as "no effect."
• Conversions are binary per scope and goal. No revenue, no continuous metrics yet, and you can't get a second conversion for the same goal from the same scope. If your KPI is order value, this release can't measure it — model it as a threshold goal or wait.
• Guest identity lives in a long-lived http-only cookie. Consent managers, aggressive cookie blocking, or third-party-cookie-pruning browsers fragment guests into multiple participants and dilute the test. The documented limitation is the same shape: sign in on a second device and that device's guest exposure stays a separate participant.
• JS conversions are session/cookie-derived, never body-supplied. That's good security, and it also means a stateless or cookieless context won't attribute — the call succeeds and counts nothing. Verify by watching exposure counts move in the same request.
• Gates default to local-only. 403 in staging is expected, not a bug. Define both gates in a provider and cache-clear before debugging anything else.
• Bot exclusion hides your own synthetic traffic. Uptime monitors, Lighthouse runs, and some QA harnesses use bot-like user agents and will generate no exposures — so "traffic but no data" can also mean "only bots hit that page."
• Long-lived workers and Octane hold stale variants. Pennant caches feature values; after a winner rollout, flush before trusting what a queue worker or Octane process reports.
• php artisan about is your cheap smoke test. If the Experiments entry is missing, the provider isn't registered and the artisan commands won't behave as expected.
• Pre-1.0, unofficial, MIT. Pin the version and read the changelog before minor bumps; the statistics surface in particular is likely to move.