Laravel Ai Headroom Laravel Package

Technical Evaluation

Architecture fit. laravel-ai-headroom is a governance layer for laravel/ai, shipped as a conventional Laravel package: a service provider, a publishable config/ai-headroom.php, a Headroom facade, an agent middleware (EnsureHeadroom), and five Artisan commands. It occupies a well-defined seam — the point where an agent step is about to be sent to a provider — and adds counting, limiting, cost estimation, calibration and event emission without persisting prompts or responses. For any application already standardising on laravel/ai for agent orchestration, this is close to a drop-in fit; for applications on other abstractions (direct SDK calls, LangChain-style orchestration, Queued jobs wrapping raw HTTP), the value is limited to the standalone Headroom::count() / Headroom::for() helpers and the model registry.

Integration feasibility. The install path is standard Composer plus config publish, with sensible environment overrides (HEADROOM_CURRENCY, HEADROOM_CACHE, HEADROOM_CALIBRATION_STORE, etc.). Enforcement is opt-in per agent via middleware, and the package explicitly supports a non-blocking mode (->report()), which makes a staged rollout feasible. Three capabilities materially lower adoption risk: Headroom::fake() with assertion helpers for the test suite, the auto counting method that avoids provider round-trips for clear passes and clear fails, and the event API (TokensCounted, TokensUsed, HeadroomLow, RequestBlocked, RequestFailed) as the integration point for observability — the package does not require us to adopt its storage. headroom:inspect exiting non-zero gives a CI gate for prompt/context regressions on day one.

Technical risk. Principal risks, in order:

1. Pre-1.0 maturity. v0.1.0 with no prior assessment baseline means no demonstrated upgrade history, no deprecation policy and no compatibility guarantees. Assume breaking changes across minor versions until 1.0.
2. Upstream coupling. The package depends on laravel/ai internals (driver names, invocation IDs, withMessages() / withTools() semantics). Release notes do not state a version constraint; upstream churn in a young first-party package is a real breakage vector.
3. Registry staleness is a correctness risk, not a cosmetic one. The shipped registry is dated "checked 2026-10-16" and warns at >60 days old. Stale prices mean cost limits enforce against wrong numbers; stale context windows can produce false passes or false blocks. Any enforcement deployment needs an owner for registry maintenance.
4. Calibration is cache-backed. Learned correction factors live in HEADROOM_CALIBRATION_STORE; a cache flush or eviction silently resets models to "still learning," widening safety margins and changing block behaviour. Concurrency behaviour of concurrent writers to the same calibration key is unstated.
5. Native counting sends prompt content to provider endpoints (count_tokens, input_tokens, countTokens). This is a data-flow consideration for any workload with prompt-confidentiality or residency constraints, and it adds latency and rate-limit surface in auto escalation bands.
6. Estimation error on non-OpenAI providers relies on per-provider estimate_ratios plus a safety margin until calibration matures — acceptable with ->report(), weaker as a hard gate.

Key questions.

• Supported PHP / Laravel / laravel/ai version matrix, and the package's stated semver policy pre-1.0?
• What is the documented fallback when a native counting endpoint fails or times out mid-auto escalation — fail open (estimate), fail closed (block), or error?
• Is EnsureHeadroom safe under concurrent queue workers, and what is the cache-write contention behaviour for calibration?
• Does calibration state need to be shared across regions/hosts, and is the chosen cache store (Redis?) assumed rather than merely supported?
• Native counting transmits prompt content — is that documented as a data-egress path, and can estimate be forced globally to avoid it?
• What is the maintenance posture: release cadence, registry refresh process, security disclosure channel, licence, and CI/test coverage?
• Is there a per-tenant or per-agent budget story beyond resolveContextUsing() tagging, or is aggregation left entirely to our own event consumers?

Integration Approach

Stack fit. Native fit for Laravel 11/12-era applications using laravel/ai agents, with Redis (or similar) available for the optional native-count cache and calibration store. No database migrations, no new persistent schema, no queue worker required. Currency support is USD/GBP via static exchange_rates config — adequate for cost visibility, not for finance-grade reporting; treat all Cost figures as approximate, and note the approximate flag when cache tokens cannot be priced.

Migration path. Greenfield — there is no prior state to migrate. The only migration-shaped work is retrofitting existing agents: each agent that should be governed needs EnsureHeadroom added to its middleware stack, and each agent's model must resolve cleanly against the registry (shipped, overridden via models, or covered by fallback_model).

Compatibility. Confirm Composer constraints before scoping. Registering counters for non-Anthropic/OpenAI/Gemini drivers via the counters map is supported but means writing and owning a counter class. The fallback_model is deliberately cautious for unknown models, which will bias toward over-counting — acceptable in report mode, needs validation before enforcement.

Sequencing. Recommended four-phase rollout:

1. Install and inventory (week 1). Composer install, publish config, pin package version exactly (no ^ pre-1.0). Run headroom:models across all providers in use; reconcile every agent's model against the registry; add models overrides for any gaps. Run headroom:inspect against a representative prompt per agent to sanity-check counting method and estimates.
2. Observe only (weeks 2–3). Attach EnsureHeadroom with ->report() everywhere, or omit limits entirely and rely on events. Wire TokensUsed, HeadroomLow, RequestBlocked and RequestFailed into existing observability, joined on TokenCount::requestId(). Use resolveContextUsing() to tag tenant_id (or equivalent) so per-tenant consumption is attributable downstream. Start Headroom::recordUsage() for any Headroom::for() pre-flight counts.
3. Calibrate (weeks 3–5). Let self-calibration converge; monitor headroom:calibration for per-model sample counts, correction factors and measured error. Do not enforce until the models carrying the most traffic are active rather than "still learning." Decide calibration.smoothing and safety_margin_floor against observed error, not defaults.
4. Enforce (week 5+). Enable blocking limits per agent, starting with the highest-cost agents. Add headroom:inspect to CI for prompt/context regression gating. Establish a recurring owner task for the 60-day registry staleness warning and for reviewing RequestFailed rates.

Operational Impact

Maintenance. Low code footprint, but two recurring obligations: (a) model registry currency — the package warns at 60 days stale, and someone must action that warning or enforcement silently degrades; (b) package upgrades — pre-1.0 means reading changelogs, pinning versions, and budgeting a small spike per minor release. Optional infrastructure (native-count cache, calibration store) adds cache-configuration ownership but no schema.

Support. Self-service surface is good: headroom:models for registry truth, headroom:inspect for per-request forensics, headroom:calibration for model health, Headroom::fake() for developer tests, and docs/events.md as the integration contract. The absence of a shipped dashboard means support depends on our own event consumers — that must be built or the package is effectively a black box in production. Add a short internal runbook covering: how to read a TokenLimitExceeded, how to inspect a blocked payload, how to reset calibration (headroom:calibration --reset --model --force) and what that does to margins.

Scaling. Counting is per-step and in-process; auto mode means the provider round-trip cost is bounded to the escalation band, which keeps steady-state latency impact small. Watch three scaling axes: native-count endpoint rate limits as concurrency grows; cache sizing/eviction under load (calibration loss on eviction is a behavioural change, not just a performance one); and event volume at high step counts — sample or aggregate before shipping everything to a log sink.

Failure modes.

• Cache store unavailable → calibration and native-count cache degrade; expect margin widening and extra provider calls. Enforcement should fail open in report mode, and the fail-closed decision for blocking mode must be made deliberately.
• Native counter endpoint down or rate-limited → auto escalation cannot complete; behaviour unstated and must be confirmed.
• Stale registry → wrong prices and context windows; cost limits mis-enforce.
• resolveContextUsing() failure → logs a warning and never fails the request (good), but silently untagged traffic breaks per-tenant attribution downstream.
• Long-running streams → TokensUsed is reported after each step including streams, so cost visibility exists, but stream abort mid-flight is worth testing against RequestFailed.
• Currency misconfiguration → GBP limits computed on a stale hardcoded exchange_rates value; budget drift with no runtime alarm.

Ramp-up. Modest for the installing engineer — Composer, config, one middleware line. Real ramp-up cost is conceptual, spread across the team: understanding the three count methods (Native, Estimated, Calibrated) and when each is in play, why an estimate can be "confident" or not, what calibration does over time, and that Cost is worst-case output plus input rounded to 8dp. Budget a short internal session plus the runbook before enforcement goes live, and expect the first two weeks in report mode to be the real learning curve.