Laravel Telemetry Laravel Package

Technical Evaluation

Architecture fit. Unchanged at the core: the package still solves the real PHP-FPM problem — shared-nothing workers cannot hold an in-process metric registry — via a shared MetricStore contract (Redis/APCu) with maintained indexes and no KEYS/SCAN. The three planes (Prometheus pull, OTLP/HTTP-JSON push, auto-instrumentation hooks) still map cleanly onto the Laravel lifecycle. v3.8.1 is a patch and adds no architecture: it repairs the queue retry span, which until now was the one place the package's own instrumentation produced a red span with nothing to read. A job attempt that threw and was released ended as an error with the literal status released for retry and no exception attached, because only a terminal JobFailed carried one. The fix sources the attempt's exception from JobExceptionOccurred on every supported Laravel version and from JobReleasedAfterException where that event carries one (Laravel 13.31+), attaching an exception event, error.type, and the exception message as span status, with explicit dedupe so a terminal failure still records one exception event per attempt rather than two. Architecturally this is small but it says two things worth recording: the package treats retryable failure as a first-class failure state, not just terminal failure; and it now ships version-gated code paths across a widening Laravel support matrix (13.31 exists and is being tracked), which is a compatibility-maintenance fact our upgrade testing must absorb. The v3.7.0 scope change — AI workload observability as an opinion — stands unchanged: derived cost from a user-supplied price table, bounded context-label promotion, and an unsampled gen_ai.client.inference.operation.details event plane that never carries content. The tail-sampling keep policy (traces.details.keep_when, traces.details.keep_spans) also stands as the answer to the previously unaddressable detail-span trimming gap.

Integration feasibility. Still low friction, and the upgrade from v3.7.x is close to free: a patch with no config change, no migration, no new surface. v3.7.0's features remain inert until configured ("nothing changes until you add prices" / "until you add rules"), events default off. So the feasibility picture is unchanged and now slightly better, because one diagnostic hole we would have had to work around — retry storms producing unreadable error spans — is closed by the vendor rather than by us. The two front-of-mind questions from the prior review remain open: whether laravel/ai is a soft/optional dependency (so non-AI services don't inherit a dependency edge into a young first-party package), and the per-span CPU cost of the keep-rule matcher (keep_spans with db.query.text => ['insert *','update *'] implies pattern evaluation against query text for every DB span). v3.8.1 adds a third, narrower one: exception messages now land in span status and error.type on every retried attempt, so span payload — not just log payload — is now a place sensitive strings can appear.

Technical risk. The vendor risk profile is unchanged in kind but keeps improving in evidence: three stars, zero dependents, single-organization namespace, bus factor ~1 — but a coherent themed minor, an opt-in posture, and now a fast, precise patch release is exactly the maintenance signal we wanted. Self-attested quality gates (PHPStan 9, Pest with Redis/APCu/benchmark groups, ADRs, changelog) remain unproven at scale. The AI risks from v3.7.0 are ours to manage and unchanged: (a) pricing drift — no price list ships, so cost metrics are only as accurate as a config file nobody owns, and an unpriced model silently yields no cost rather than a wrong one; (b) cost semantics — cost is derived from observed token counts, not provider invoices, so cache-tier discounting and provider rounding diverge unboundedly without reconciliation; (c) currency as unit — flipping TELEMETRY_AI_CURRENCY changes series identity, fragmenting the counter in a mixed-currency estate; (d) cardinality via context — ai_metric_labels promotes Telemetry::context() keys to Prometheus labels and the runtime still does not enforce boundedness, so user/tenant IDs are a live footgun; (e) PII in observability — user.id now lands on spans and, with events enabled, in log records, pulling the backend into privacy/retention scope; (f) log volume — unsampled per-call records scale linearly with inference volume; (g) rule sprawl — keep rules are additive and unversioned, so trace-storage spend inflates quietly. v3.8.1 introduces two small risks of its own: (h) exception content in span status — retried attempts now replicate the exception message into telemetry, and connection/validation exceptions routinely carry DSNs, SQL fragments, IDs or user-supplied text; this is a smaller surface than the usage-event log plane but it is a surface we did not have before; (i) version-matrix divergence — on Laravel <13.31 a released-after-exception attempt may still fall back to the bare released for retry status, so the same failure renders differently across our services, and any alert keyed on exception presence rather than on retry/release state will fire inconsistently. Prior risks persist: PHP ^8.3 hard floor, X-Telemetry-Label / telemetry:watch as a header-reachable control plane needing authz review, and curl-on-terminate tail latency.

Key questions.

1. Is laravel/ai a required or suggested dependency, and which versions are supported? Does a service without LLM calls pay anything in install weight or boot cost?
2. Who owns instrument.ai_pricing.models internally, on what refresh cadence, and against what source of truth? What tolerance do we accept between gen_ai.client.cost and the provider invoice before it stops being trustworthy?
3. What happens observably when a model is unpriced — silence only, or a warning/self-metric we can alert on? Same question for a partially priced agent run (sum is only emitted when every call was priced).
4. Is currency change a supported operation, and does the unit appear in the metric name such that a USD→EUR switch creates a new series rather than relabelling?
5. What bounds exist on instrument.ai_metric_labels? Is there validation, a cardinality warning at config load, or purely documentation?
6. For ai_usage_events: what is the expected record volume per 1k calls, and does user.id in logs require a documented retention/DPA position before we enable it?
7. What is the per-trace cost of evaluating keep_when / keep_spans patterns, and is pattern matching compiled or evaluated per span?
8. (Partly answered by v3.8.1) What is the full supported Laravel matrix? The release names JobReleasedAfterException as Laravel 13.31+ and claims JobExceptionOccurred coverage on "every supported Laravel version" — confirm the exact set (10/11/12/13) rather than inferring it, and whether the release-after-exception parity gap is documented.
9. (New) Is exception message redaction, truncation, or opt-out supported before it lands in span status and error.type? If not, is that accepted as-is?
10. (Carried) PHP ^8.3 — deliberate floor or incidental? Exporter-failure self-telemetry? Published benchmark figure? X-Telemetry-Label authz gating?

Integration Approach

Stack fit. Unchanged baseline: conventional Laravel (FPM or Octane), Redis already present, Prometheus scrape target, optional OTLP backend (Tempo/Jaeger/Grafana). The release adds a conditional fit: value from v3.7.0's AI features requires (a) laravel/ai in use, (b) a resolved authenticated user where per-user attribution is wanted, (c) a price table we author, and — only for exact per-user sums — (d) a log pipeline that can absorb unsampled per-call records. Services meeting (a) but not (d) can still get cost and token metrics; they just cannot get exact attribution from logs and should not pretend otherwise. Where no OTLP endpoint exists, tracing remains off, and that now also disables the per-call usage-event plane's natural home — and, relevant to v3.8.1, disables the queue-span exception detail too, so a Laravel-only metrics deployment sees retries only as counters and latency, not as diagnosable spans. Worth stating explicitly in the fit criteria: queue-heavy Laravel services benefit from tracing even with no AI workload, and that argument got stronger with this patch.

Migration path. v3.7.0 → v3.8.1 is a patch bump: no config change, no data migration, no config schema break, no new settings. The three entry states from the prior assessment still hold (no observability / Pulse-Telescope / OTel SDK already deployed), and the "do not run both against the same backends" rule still applies — the gen_ai.* conventions are the OTel GenAI ones, so if we already run the OTel SDK for AI instrumentation we must not double-emit cost and usage series. Exit cost stays low: the facade wrapper (counter, histogram, gauge, span, event) plus the thin cost()/pricing adapter keeps a future swap to one file. The one new lock-in of substance is the configuration vocabulary (instrument.ai_pricing.models shape, keep-rule DSL) — cheap to re-express, but worth a note in the wrapper. The only behavioural change in this release is in our favour (retry spans become readable), which means the migration note is one line: re-baseline any alert or dashboard that keyed on the literal released for retry status string.

Compatibility. Verify: our Laravel versions relative to 13.31, because release-after-exception parity differs across that boundary and we will likely run a mixed estate; laravel/ai presence/version and whether it is optional; ext-redis vs predis; APCu availability; Octane/Swoole behaviour (the single-writer ADR implies support — confirm); Horizon/queue middleware conflicts, now including whether a custom JobExceptionOccurred listener or a queue middleware interferes with the exception capture path; and that our log pipeline's sampler does not sit downstream in a way that drops the "unsampled" usage events anyway, which would silently break exactness. Also confirm the AI span attributes we intend to alert on (gen_ai.agent.name, user.id) are populated in our auth flow, since user.id only appears when a user was already resolved. Add one check specific to this release: whether exception-class filtering or message sanitisation is available, and if not, whether our span backend's retention and access controls are acceptable for raw exception text.

Sequencing.

1. Week 0 — spike + upgrade. One non-production service. Bump to v3.8.1 with no new config, confirm zero behavioural change apart from retry-span detail, capture install weight and baseline latency.
2. Week 0.5 — retry-span verification (new). Deliberately fail and release a job in staging; confirm the exception event, error.type, and status message appear, confirm a terminal failure records exactly one exception event per attempt (the double-count is explicitly called out as a regression to avoid), and update any alert that matched on released for retry.
3. Week 1 — overhead gate. Run the benchmark group plus our own load test at p50/p99, including a run with representative keep-rules enabled so the per-span pattern-matching cost is measured, not assumed; include a retry-storm scenario so exception-event volume is measured too. Go/no-go.
4. Week 2 — instrumentation discipline. Bounded label vocabulary (route pattern, queue, status, region) published before anyone writes a counter. Extend the same one-pager to cover ai_metric_labels: explicitly forbid user/tenant IDs as metric labels; allow plan and region only.
5. Week 3 — AI cost pilot. Author a starter price table for the two or three models actually in use, reconcile a week of gen_ai.client.cost against provider billing, and document the delta. Decide unpriced-model behaviour (alert vs ignore). Do not enable ai_usage_events yet.
6. Week 4 — traces in shadow mode at 0.01–0.05 sampling, one backend, dashboards built on root-span tallies. Add a first, narrow keep-rule set (agent invocations; checkout route) with a named owner and a review date.
7. Week 5+ — usage events + selective sampling. Enable ai_usage_events only after privacy/retention sign-off on user.id in logs and a volume estimate. Raise sampling via directives, per-route, not globally.
8. Deferred — native profiler (cboxdk/telemetry-native, C extension → build/distribution plan) and LogHandler; separate change requests.

Operational Impact

Maintenance. Dependency surface stays small and there are still no compiled artifacts in the default path, so upgrade cost is low — and the vendor's posture of shipping inert-by-default features plus fast, precisely scoped patches is the upgrade behaviour we want to reward with prompt adoption. The maintenance burden that does grow is config-shaped and human: a price table that must track provider pricing changes (including cache-read/write tiers, which providers do reprice), a currency setting, a context-label allowlist, and a set of keep rules that will accumulate. Treat all four as owned configuration with a review cadence, not as one-time install steps. New this release: a version-conditional behaviour in the queue path means our upgrade checklist must include a retry scenario whenever we move Laravel minor versions, since the amount of detail attached to a released attempt depends on the framework version. Upstream-abandonment mitigation is unchanged: pin, retain fork rights, keep the internal wrapper thin. The changelog remains a reliable signal, and a same-week patch with an honest root-cause paragraph strengthens that read.

Support. Documentation depth is still the strongest non-code asset, and the release notes are unusually honest about semantics — including this one, which states the before-behaviour, the fallback string, the dedupe rule, and the version gate. What is not covered and we must write ourselves: the pricing-reconciliation procedure, the cardinality rule for ai_metric_labels, the meaning of a missing cost, the keep-rule review process, and now what exception text is allowed into spans. Still no community, no issue history, no third-party case studies — plan for "we are the support" with one named owner, now with two hats beyond plumbing: cost-metric integrity and telemetry content hygiene.

Scaling. Three hot paths now instead of two. Redis remains the per-request write target and cardinality, not volume, is the killer — ai_metric_labels widens that attack surface by inviting tenant-ish keys onto gen_ai.client.*. Prometheus scrape-time rendering is CPU on an FPM worker and should be pinned to a dedicated endpoint. Unsampled usage-event log volume scales linearly with inference calls, with no sampling lever by design, so a popular AI feature is a predictable logging bill and a potential pipeline saturation point — cap it, or route it to a separate pipeline from application logs. Keep-rule pattern matching adds per-span CPU on the request path; keep rule sets small and review them when trace storage grows. This release adds marginal span payload on retries: one exception event plus status per attempt, bounded by the job's retry policy, but a poison job with ten tries across many workers multiplies the volume and ships a stack trace each time — worth a span-payload budget check if retry storms are already common.

Failure modes. The package's fail-safe posture is unchanged — telemetry failures are silent by design, which is right for the request path and wrong for operations unless compensated. Required controls: alert on scrape-target absence/stale series; alert on export failure rate (or a synthetic canary span if no self-metric exists); panel telemetry.overhead_ms; Redis memory and per-metric cardinality budget with a hard cap. v3.8.1 removes one failure mode we would otherwise have carried: a retried job is no longer an opaque red span with no exception attached, so a retry storm is now diagnosable from traces without correlating to application logs. It adds two: exception-message spill, where raw exception text — connection strings, SQL, IDs, user-supplied input — reaches span status and the observability backend, so that backend's access control and retention become part of our data-handling scope; and cross-version rendering divergence, where an attempt released after an exception looks detailed on Laravel ≥13.31 and bare on earlier versions, so alerting should key on retry/release state rather than on the presence of an exception. Carried failure modes to document: silent zero cost for an unpriced or newly released model (and for an agent run where even one call was unpriced, since the sum is only emitted when all calls are priced); currency flip fragmenting the cost counter; usage-event loss upstream of the package if the log pipeline samples, quietly destroying the exactness guarantee; PII spill via user.id into a backend outside retention scope; keep-rule sprawl inflating trace storage without a visible change in request count; and the standing ones — APCu losing metrics after FPM restart, telemetry:watch left running, web/queue config drift producing inconsistent series.

Ramp-up. A Laravel-fluent engineer still reaches working metrics in half a day to a day, and a useful dashboard set in one to two weeks. Add roughly half a day to a day for the AI cost path: authoring prices, validating against an invoice, and agreeing the unbounded-label rule. Tail keep rules are an hour to learn and a quarter to govern. Queue-span semantics are now cheap to teach rather than a known blind spot — the earlier caveat "a red span with nothing to read" no longer needs explaining. The packaged skills and ADRs continue to compress onboarding versus a from-scratch OTel SDK integration. Budget one engineer at ~25% for a month for a single-service pilot, plus a finance/FinOps touchpoint if cost attribution is a stated goal; org-wide standardisation remains a separate decision gated on the overhead benchmark, the PHP 8.3 floor, the confirmed Laravel support matrix, and who owns the price table.