Experiment Laravel Package

Technical Evaluation

Architecture fit. The package occupies the assignment layer of an experimentation stack: declarative experiment definitions, deterministic hash-based bucketing, a facade for resolution, Blade helpers, and goal/conversion counters. For a Laravel-native product this is a clean fit — assignment happens in-process, requires no external service, and keeps experiment configuration in code (reviewable, testable, cheap to retire). The critical architectural gap is that it stops at assignment. The measurement layer — exposure logging, goal deduplication, bot filtering, attribution windows, reporting, and statistical inference — remains ours. That split is often correct for a small team that wants the mechanism in-house, and clearly wrong for a team that expects experimentation rigor out of the box. This package forces that conversation early, which is its most valuable property as a candidate.

Integration feasibility. Mechanically straightforward: composer require orchid/experiment, publish the config, declare weighted variants, resolve the variant once at the edge (middleware/controller), branch in the view, record conversions through one seam. Footprint is small; the facade is the contract. The blocker is not code, it's knowledge: documentation is thin, the only public release signal is 2.4 ("Laravel 9 support"), and the supported Laravel/PHP matrix is effectively unknown until we read composer.json and diff the PR. Budget a discovery spike before committing to a roadmap that depends on this.

Technical risk. The dominant risk is maintenance, not functionality. Zero dependents, an overall score of 0.3, an opportunity score of ~20, sparse release notes, and a latest release that only widened support to Laravel 9 (itself past security support) together describe a package with a low probability of tracking future Laravel majors. Secondary risks are silent readout invalidation: session/cache-based assignment flipping on cache clear, hash reshuffling when weights change mid-flight, bot-inflated exposures, and ambiguous goal deduplication. Each is fixable, but only if we know which mode the package runs in before experiment #6.

Key questions.

• What is the verified Laravel/PHP support matrix — and are we on it, or does this require a fork?
• Is variant assignment derived from identity each request, or remembered in session/cache? The former is cache-flush-proof by construction.
• Where are exposures and goals recorded, and can we add columns (context, dedupe key, user-agent) without patching the package?
• Does the shipped schema enforce a unique constraint on (experiment, goal, identity), or must reporting SQL dedupe itself?
• Who owns the fork if upstream stalls on a Laravel major, and what is the acceptable time-to-patch before our upgrade path is blocked?
• Is a hosted experiment/flag service cheaper than the measurement layer we'd be building around this?

Integration Approach

Stack fit. Laravel-native by intent, and compatible in spirit with Blade, Inertia/Livewire, and JSON APIs. The one rule that matters: the variant is resolved server-side and passed down as a value; the client never picks a variant. That preserves stickiness, avoids render flicker, and keeps the readout clean. Wrap the package behind a thin internal interface — resolve(string $experiment, $identity): string and converge(string $experiment, string $goal, $identity): void — so the package is an implementation detail, not an API every controller learns.

Migration path. Greenfield adoption is a two-hour integration for one experiment; the real migration is organizational. If manual flag config or ad-hoc database flags exist today, migrate them behind the same resolver interface so callsites don't change twice. If we're on Laravel 10+, plan the fork as part of adoption, not after — fork the repo, pin to a commit, and treat the fork as the artifact we support. Upgrading 2.3 → 2.4 should be verified by diffing migrations and the service provider, not by trusting the "Laravel 9 support" changelog line.

Compatibility. Validate in staging against php artisan config:cache, route:cache, and package discovery before production. Config-driven experiment packages commonly read config at boot; a cached config with stale definitions will not pick up new experiments until recached — a footgun that looks like a bug in the experiment. Inspect any shipped migration and treat the resulting tables as our reporting schema, because they will be.

Sequencing.

1. Version-matrix spike — read composer.json, the service provider, and the migrations directory. Decide adopt / fork / reject. Cap it at a day.
2. One experiment end to end — a single landing-CTA test, control/treatment 50/50, resolved in the controller, branched in Blade, one conversion recorded on submit. This surfaces stickiness and deduplication before any roadmap decision depends on the tooling.
3. Reporting from day one — build the per-experiment saved query or artisan report (exposures, conversions, conversion rate, filtered by exposure date range) during experiment #1, not after #4.
4. Identity function decision, written down — user_id for authenticated, cookie/session ID for guests, with the resolved variant stored alongside for audit and a test asserting variant stability across logout/login.
5. Wrap and fake — internal resolver interface plus a force()/fake() implementation so every variant branch is unit-testable without depending on hash behavior.
6. Then widen — feature gates as 0/100 or 10/90 weights, kept in a separate config namespace from experiments so gates aren't counted as exposures and tests aren't flipped manually mid-flight.
7. Lifecycle discipline — declare every experiment running, stopped (removed from config, data retained), or shipped (winning variant becomes the only code path, branch deleted). Retire aggressively; stale experiments keep assigning traffic and inflating exposure counts.

Operational Impact

Maintenance. The maintenance burden is ours, not the maintainer's. Cap exposure by keeping the package strictly behind the resolver interface, pinning the version, and owning a fork when necessary. The engineering cost is small; the analytical cost — dedupe, bots, attribution, reporting — is the ongoing line item. Estimate it as a standing platform-team responsibility, not a one-time integration.

Support. Assume self-support. Public signals (zero dependents, a first-time contributor shipping the only recent change, no visible issue activity) suggest no reliable upstream. Documentation is a release note and a thin README; the source is the contract. Plan for a named internal owner, a private fork, and a short patch SLA for Laravel major bumps.

Scaling. Assignment itself scales trivially: hash-based bucketing is O(1) per request and cache-friendly. The scaling pressure lands on the data layer — conversion tables grow monotonically, and reporting queries need indexing, date-range filtering, and explicit DISTINCT if the schema lacks a uniqueness constraint. At high traffic, the cost is your database and queries, not the resolver, and that's true of any in-house experimentation layer.

Failure modes to watch.

• Cache clear flips variants if assignment is remembered rather than derived — the quiet killer of in-flight tests.
• Mid-flight weight changes reshuffle the hash space and move existing visitors between buckets. Freeze weights at launch; end and rename rather than edit.
• Bot traffic inflates exposures, depresses conversion rate on both variants, and mimics a failing test.
• Duplicate conversions from double submits, refresh, or cross-device sessions — invisible in totals, obvious when segmented.
• Stale config cache silently drops newly declared experiments.
• Typo'd experiment names split traffic across two underpowered buckets; centralize identifiers in an enum.
• Client-side variant selection destroys stickiness and introduces flicker.
• Upstream abandonment blocks a Laravel upgrade — mitigated only by the interface seam and a fork.

Ramp-up. Low for engineers (config plus a facade), high for the organization's decision discipline. The package makes shipping a test a two-hour task but supplies no dashboards, no significance testing, and no sample-size guardrails. PMs must own the pre-registered minimum runtime and sample size, resist daily peeking, and run a review-and-decide ritual — otherwise the tooling lowers activation energy for tests whose results nobody is willing to act on. Realistic ramp: one engineer productive in a day, the team trustworthy in a couple of experiments, provided the reporting and dedupe seams are built during experiment #1 rather than retrofitted later.