Experiment Laravel Package

Getting Started

Verify the compatibility matrix before anything else. The public record for this package is thin: the only published release note (2.4) says exactly one thing — "Laravel 9 support." Treat the supported Laravel/PHP range as unknown until you read the package's own composer.json require block against your app's version. If you are on Laravel 10 or 11, plan for a fork or a pinned-back app before you plan for experiments.

Install and configure:

1. composer require orchid/experiment (note the vendor is orchid, the source repo is tabuna/experiment — search results and GitHub paths disagree, which is the first hint that docs are sparse).
2. Publish the package config (php artisan vendor:publish and pick the experiment provider/tag from the list if the tag isn't documented).
3. Run any migrations the package ships and read the resulting schema — if it creates exposure/goal tables, those tables are now your reporting schema and you need to know their exact columns.
4. Read the service provider to see what's bound in the container and whether the storage driver is swappable in config.

Where to look first, in this order: the published config file (it is where experiments and weighted variants are declared — this is the real API surface) → the facade's public methods (the assignment seam) → the Blade helper(s) → any tracking model or migration. Budget twenty minutes of source reading. The README is not the contract here; the config shape and the facade signatures are.

First use case — one experiment, end to end. Resist the urge to launch five. Ship a single landing_cta test with two 50/50 variants:

• Declare landing_cta with variants control and treatment in config.
• In the landing route's controller (or a small middleware), resolve the variant once and pass it to the view.
• Branch in Blade on the passed value.
• On the thank-you/submit route, record one conversion for that experiment.
• Write a one-off SQL query that prints exposures and conversions per variant.

The reason for starting with one is not simplicity — it's that a single experiment end-to-end surfaces the two failure modes that silently invalidate every readout you'll ever produce: stickiness (does the same visitor stay in the same bucket?) and deduplication (does one visitor submitting twice count twice?). Both are invisible until you have data, and both are far cheaper to solve on experiment #1.

Implementation Patterns

Centralize experiment names in an enum or constants class. Never type an experiment string at a call site. In a config-driven package, a typo either throws at resolve time or — worse — silently resolves as a fresh, empty experiment and splits your traffic between two underpowered buckets that each look inconclusive.

Resolve once per request, at the edge, then pass down. Do the resolution in middleware or the controller action. Not in a Blade partial, not in a model accessor, not in a view composer that may execute twice.

• Blade: branch on the value your controller passed (@if ($variant === 'treatment')). If you use the package's Blade helper directly, first confirm it reads the same cached assignment rather than re-hashing — otherwise you can get a different variant on the same page.
• Inertia / Livewire / SPA: pass the variant as a prop or public property. Never let the client choose — client-side assignment destroys stickiness and causes render flicker, both of which corrupt the readout and annoy users.
• API: include the variant in the response payload so the front end has nothing to decide.

Funnel every conversion through one seam. Build a small TrackConversion service — or a listener on an ExperimentConverged-style event — and have every goal in the app fire through it. This is where deduplication, bot filtering, and extra attribution columns will live later. It is also the seam that makes replacing the package a one-file change instead of a grep across every controller.

Choose your identity function explicitly and write it down. Hash-based bucketing needs a stable seed:

• Authenticated user → user_id. Stable across devices, but a visitor who starts as a guest and later logs in can flip buckets unless you re-resolve carefully.
• Guest → first-party cookie or session ID. Stable per browser; gone on cookie clear; different per device.

Write a test that logs a user out and back in and asserts the variant is unchanged. That is the single most common way in-app experimentation quietly breaks.

Reuse the same primitives for feature flags — in a separate namespace. A 0/100 weighting is a rollout gate; 10/90 is progressive exposure. This doubles the package's value, but keep gates and tests in separate config blocks, because gates should never be recorded as exposures and tests should never be flipped by hand mid-flight.

Force variants in tests via your own interface. Wrap the facade in something like ExperimentResolver with resolve(string $experiment, $identity): string and converge(...), and give the test suite a fake that returns whatever variant you specify. Every branch becomes deterministic, and you gain a clean exit path if upstream stalls on a Laravel major.

Manage the lifecycle. Every experiment is running (in config, assigning traffic), stopped (removed from config, data retained), or shipped (winner is now the only code path, branch deleted). Experiments left in config keep assigning and keep inflating exposure counts, which makes old reporting numbers unreproducible.

Reporting is SQL you write, not a dashboard you buy. Build the saved query during experiment #1: exposures per variant, conversions per variant, conversion rate, and a date range filter so you can align with the launch date. There is no built-in analytics UI.

Gotchas and Tips

Cap your exposure with an internal interface. Zero dependents, sparse release notes, and a latest release that only widened support to Laravel 9 are signals to keep this package behind a thin wrapper class. Two methods of your own (resolve, converge) turn a future fork or a swap to a hosted service into one file changed instead of a refactor.

Test under php artisan config:cache and route:cache. Config-driven experiment packages frequently read config at boot. A cached config with stale experiment definitions will keep assigning the old variants until you recache — and this will happen during a deploy, not in dev.

Know whether assignment is derived or stored. If variants are remembered in session or cache rather than derived deterministically from identity each request, then cache:clear or a session regeneration reassigns live visitors mid-test. Derived-on-the-fly is cache-flush-proof by construction; check which mode you're in and test a cache clear against a live-ish staging environment.

Freeze weights once traffic starts. Changing weights or adding a third variant reshuffles the hash space and moves existing visitors between buckets. If you must change, end the experiment and start a new one under a new name — your data stays clean and your PM stays confused for only five minutes.

Decide deduplication semantics per goal, and enforce them yourself. "Purchase" is once per order; "viewed pricing" is probably once per session; "clicked CTA" might be every occurrence. The package doesn't know your product. Getting this wrong is invisible in the total and glaringly obvious the first time you segment.

Filter bots in your conversion seam. Nothing in the description suggests crawler filtering. Unfiltered bot traffic produces near-zero conversion rates on both variants and looks exactly like a losing test.

Confirm the schema before committing to a reporting design. Check whether there's a unique constraint on (experiment, goal, identity). If there isn't, your reporting SQL must do the DISTINCT itself — and you must be explicit about which timestamp anchors exposure-to-conversion attribution.

Build debugging affordances early. Add an artisan command like experiment:resolve {experiment} {identity}, a staging-only response header exposing the resolved variant, and a log line that puts the variant into the request context so error reports carry it. When a PM says "the treatment looks broken," you want a two-second answer to "was that user actually in treatment?"

Inspect extension points, not just the API. Before adopting, identify three things: where exposures are recorded, where goals are recorded, and whether you can add columns to the tracking tables without patching the package. Those three facts matter more than how pleasant the Blade helper is — they determine whether this survives real reporting requirements or becomes a fork on day thirty.

Upgrading 2.3 → 2.4: read the PR, not the changelog. Compatibility-only releases usually carry no migrations, but "usually" is not "verified." Diff the migrations directory and the service provider before assuming a drop-in bump.

Sample-size discipline stays yours. The package splits traffic; it says nothing about whether your result is real. Agree on a minimum runtime and sample size before launch, and resist reading the numbers daily. Peeking at an underpowered test is how teams ship noise as a decision.