Ab Laravel Package

Technical Evaluation

Architecture fit

• jenssegers/ab occupies a narrow, application-layer seam in a Laravel codebase: define an experiment, resolve a variant for a visitor, persist that assignment, record a conversion. No queue workers, no external service, no infrastructure footprint beyond whatever storage it selects for sticky bucketing.
• That small blast radius is its principal virtue from a TPM seat — it is a component, not a platform commitment. It can be adopted, evaluated, and removed without touching the deployment topology.
• Fit is conditional on framework baseline. The only capability signal in the supplied material is v0.2.0 adding Laravel 5 support. That makes it a candidate only for estates deliberately pinned to Laravel 5.x. On a modern-Laravel estate the release adds nothing, and the absence of any evidence about 6+/8+/10+/11+ support is itself the finding.
• Fit is also conditional on criticality. Server-side A/B testing is non-differentiating for most products and differentiating only for growth-led consumer products where experimentation velocity is the product. That classification must be forced before scoping anything.
• The correctness semantics that actually matter — sticky assignment, goal attribution, exposure logging — are the least documented parts. A one-line changelog is not an architecture document, and the gap should be treated as a discovery cost to be budgeted, not a footnote.

Integration feasibility

• Mechanically easy. Composer install, service-provider registration (manual wiring in config/app.php pre-5.5, auto-discovery from 5.5), optional config publishing, possibly a migration for the assignment/goal store. A competent Laravel developer gets a spike running in under a day.
• The real work is trust establishment, not integration: does assignment stay sticky across sessions, cache evictions, and devices? Does a goal fire once per visitor or once per request? Is bucketing deterministic and evenly distributed? None of that is answerable from the supplied sources — only from reading the source and running experiments.
• Ecosystem corroboration is absent: zero stars, zero dependents, no prior community assessment or third-party write-up. Not disqualifying for an internal tool, but it means we are the first serious adopter and we absorb the entire discovery cost.
• Because the package covers only the cheap half of experimentation — assignment and goal hooks — integration effort should be scoped against the full capability. Significance testing, sample-size planning, guardrail metrics, and dashboards remain ours to build or buy regardless of this decision.

Technical risk

• High overall. Drivers: pre-1.0 (minor bumps may legally break), an end-of-life framework baseline, no published test/CI evidence, no compatibility matrix, no stated maintenance or security-disclosure posture, and a single-release evidence trail.
• Data-integrity risk is the one that matters commercially. A broken experiment framework does not crash — it silently reports the wrong winner, and roadmap decisions get made on corrupted numbers weeks later. This is a measurement instrument, not a utility; the trust budget must be calibrated accordingly.
• Baseline-coupling risk. Adopting it creates an implicit argument for staying on Laravel 5, itself EOL. The dependency can become the reason a legacy upgrade keeps slipping — a testing tool quietly turning into a migration blocker. Force that trade-off into the open rather than letting it die in a dependency ticket.
• Secondary: if Laravel 5 support was achieved by widening or branching framework constraints, hypothetical existing adopters on newer versions may have been exposed. Verify in composer.json; do not assume.

Key questions

1. What is the exact supported Laravel/PHP matrix, and is it enforced in CI at both the lowest and highest supported minor?
2. Where does sticky assignment live — cookie, session, cache, or database? What is its TTL, and what happens on cache flush, cookie expiry, or a new device?
3. Is bucketing deterministic and stable, or does it depend on mutable inputs (experiment name, variant order, config edits)?
4. How are goals recorded — once per visitor per experiment, once per session, or once per event? What is the attribution window?
5. What is the behavior when an experiment is paused, renamed, or deleted mid-flight? Are assignments preserved or re-randomized?
6. Is bot/crawler traffic excluded from assignment and conversion counts?
7. Who maintains this, at what cadence, and with what security-disclosure process?
8. Is the package still installable and tested on current Laravel, or has it effectively frozen at the 5.x line?

Integration Approach

Stack fit

• Direct fit for a Laravel 5.x application, an internal tool, or a time-boxed prototype where the alternative is building variant assignment ourselves.
• No fit for a modern-Laravel production surface, a revenue-critical experiment program, or any code path adjacent to auth, payments, entitlements, or PII.
• Appropriate where the experiment surface is low blast radius: marketing copy, layout, button wording, an internal workflow, a prototype feature. A wrong answer costs a week, not a quarter.
• Appropriate where an analytics destination already exists. The package fires exposure and conversion events; stats, dashboards, and significance testing are ours to own.
• Not appropriate where audience targeting, RBAC, approval workflow, audit trails, consent management, or shared metric definitions across teams are requirements. That is a platform purchase, not a package install.
• The pragmatic framing: adopt it as a learning instrument — does the product team actually run experiments and act on the results? — before committing to any experimentation platform. Many teams discover they need the discipline far more than the tooling, and a cheap trial surfaces that at low cost.

Migration path

• There is no meaningful inbound migration; the outbound migration is the one to design before the first line of integration. Put every call site behind a thin internal contract (ExperimentService with variant(), track(), active()) so the package sits at the edge of a single adapter.
• That adapter boundary does two jobs: it makes a swap to Laravel Pennant, Unleash, LaunchDarkly, or an in-house flag layer a contained change, and it lets us enforce policy centrally (bot filtering, exposure logging, kill switches) regardless of what the dependency does.
• Plan the framework exit on the same timeline. If the application cannot move past Laravel 5, the package decision is downstream of a much larger platform problem and should be labelled that way in the roadmap rather than buried in a dependency ticket.

Compatibility

• Pin the exact 0.x version. Pre-1.0 minor bumps are not guaranteed safe and the changelog will not tell us what broke — add a dependency-review gate so a routine composer update cannot silently move it.
• Verify the real constraint in composer.json, not the release note. Confirm the PHP floor against our runtime.
• Check registration mechanics against our Laravel minor — pre-5.5 manual provider registration versus auto-discovery changes both the setup instructions and the rollback story.
• Confirm behavior under the app's actual session and cache drivers, not the defaults. Sticky assignment that happens to work on file sessions may behave differently on Redis with eviction policies.
• Assume dated assumptions around service providers, config publishing, and facades given the Laravel 5 baseline; budget a small amount of integration friction rather than treating the README as authoritative.

Sequencing

1. Spike (≈1 week). Read the source; answer the eight key questions; run a throwaway experiment end-to-end. Deliverable is a written findings memo, not working code.
2. Contract first. Land ExperimentService and the adapter with a contract test asserting "same visitor, same variant" across sessions, devices, and cache flushes. Nothing else merges until this is green.
3. Instrumentation validation. Run one internal, zero-stakes experiment and reconcile exposure counts and conversions against an independent source. If the numbers do not reconcile, stop.
4. Bounded production trial. Two to three low-stakes experiments on non-revenue surfaces, version-pinned, with a documented kill switch and an explicit statement that results are directional only.
5. Decision gate. Build-vs-buy recommendation backed by real experiment output. Expect the answer to favour Laravel Pennant or a thin in-house flag layer once actual requirements are known.
6. Exit criteria defined up front. If the framework upgrade lands, the package is replaced as part of that work, not after it.

Operational Impact

Maintenance

• Dependency ownership must be explicit: one named engineer owns the adapter, the version pin, and the upgrade conversation. Unowned measurement dependencies rot silently.
• Pre-1.0 plus a single-release changelog means we effectively own the code we depend on. Budget for reading source on each bump, and be prepared to vendor or fork if upstream goes quiet.
• Zero dependents in the wild is a leading indicator, not a footnote — it means upstream receives little bug pressure and edge cases we hit are likely novel to the maintainer too.
• Documentation maintenance falls to us: the sticky-assignment and goal-attribution semantics we discover during the spike must be written down in our own repo, because the package will not document them.

Support

• No stated maintenance cadence, no security-disclosure process, no compatibility matrix, no evident CI. Support is the community of one — us.
• Supportability inside the org is good if the adapter boundary holds: any engineer can answer "what variant did this user see" without knowing the package exists.
• Escalation path for a data-integrity bug is fork-and-patch, not vendor ticket. That is acceptable for internal surfaces and unacceptable for anything a board deck cites.
• Incident response must include a documented behavior for "experiment is producing bad data": pause, invalidate assignments, and state the blast radius. Define this before the first experiment, not during the first incident.

Scaling

• Technically trivial at our projected volume: one team, a handful of concurrent experiments. No infrastructure to scale.
• Organizational scaling is the real ceiling. The moment a second team wants self-serve experiments with shared metric definitions and a warehouse-native assignment log, the package is the wrong shape and a platform conversation starts. Plan for that transition rather than being surprised by it.
• Assignment storage growth is modest but unbounded — no retention policy is described, so we own pruning. Confirm the storage medium's growth profile during the spike.
• Analytics cost scales with exposure volume and is entirely ours: event volume, warehouse ingestion, and query cost are not the package's concern and should be estimated separately.

Failure modes

• Silent wrong winner (highest severity). Broken cookie/session handling flips visitors between variants; conversion rates drift; a roadmap call gets made on garbage. Detection requires an independent validation check, not the package's own reporting.
• Unstable bucketing. If assignment depends on mutable inputs — experiment name, variant order, config edits — a routine config change re-randomizes the population mid-flight and contaminates the entire window.
• Goal double-counting or under-counting. Ambiguity in whether goals fire per visitor, per session, or per event inflates or deflates lift and is invisible without reconciliation against a second source.
• Mid-flight experiment mutation. Pausing, renaming, or deleting an experiment may preserve or re-randomize assignments; either behavior is defensible, but undocumented behavior is not.
• Bot contamination. Without crawler exclusion, exposure and conversion counts include non-human traffic and skew low-volume tests.
• Dependency drift. An unpinned composer update moves the package or a transitive dependency and changes assignment behavior with no changelog entry.
• Legacy lock-in. The package quietly becomes a reason not to migrate off an EOL framework — the slowest-moving failure and the most expensive.
• Consent exposure. Cookie/local-storage assignment intersects GDPR/CCPA and the package provides no consent story; that obligation lands entirely on us.

Ramp-up

• Developer ramp is under a day for a Laravel-competent engineer: install, register, define two variants, track a goal.
• Analyst ramp is the larger effort and is not supplied by the package. Someone must own metric definitions, sample-size intuition, and the discipline to not call a winner on day two. Without that role, the tool produces confident-looking numbers with no interpretive layer.
• Stakeholder ramp requires expectation-setting up front: this is a bucketing mechanism, not an experimentation platform. The phrase "A/B testing tool" will be heard as "we have an experimentation practice"; correct that in the first communication.
• Written prerequisites before the first experiment: what counts as a conversion, how long a test runs, who reads the result, and what decision the result is allowed to inform. The package does not enforce any of this and will happily run an invalid test.

Bottom line: A low-cost, low-blast-radius component that solves the cheap half of experimentation and none of the expensive half. Approve a bounded, version-pinned, adapter-wrapped trial on internal and non-revenue surfaces only; decline for anything a board deck will cite. The decision this actually forces is not "which A/B library" but "do we intend to run experiments on a recurring cadence at all" — answer that first, and the dependency question mostly answers itself.