Laravel Haystack Laravel Package

Technical Evaluation

Release under review: v2.2.0 — "Do Not Delete Haystack On Failure" (#86)

This is a single-commit patch release from a first-time contributor. It changes failure semantics: previously, a haystack (the package's unit of work — an ordered set of jobs with shared state and a catch handler) could be removed from its persistence store when it failed. v2.2.0 preserves the haystack record on failure.

Architecture fit

• Laravel Haystack is a thin orchestration layer over Laravel's queue system, offering sequential job chains with a catch callback, shared data bag, and persisted progress — the gaps Laravel's native Bus::chain() and Bus::batch() leave around mid-chain failure inspection and resumption.
• The fix aligns the package with how TPMs and on-call engineers expect failure artifacts to behave: a failed unit of work should remain observable. Deleting state on failure made post-mortems and retries depend on log correlation, which is fragile.
• Fit is good for teams already on Redis/database queues who want durable multi-step workflows without adopting a full workflow engine (Temporal, Laravel Workflow, etc.).

Integration feasibility

• Drop-in patch upgrade from v2.1.0. No API surface change, no config keys added, no migration implied by the notes.
• Because the change is behavioural rather than structural, integration effort is near zero — but the observable behaviour of existing failure paths will change, which is the real integration consideration.

Technical risk

• Low code risk, non-zero operational risk. This is a semantic change disguised as a bug fix: systems that implicitly relied on failed haystacks being cleaned up (cache/database growth, idempotency assumptions, alerting keyed on "record disappears = failed") will now see retained records.
• Unbounded retention is the main new risk. If failures were previously self-cleaning, a persistent failure loop could now accumulate rows/keys. Retention policy and pruning must be confirmed as the package owner's responsibility or ours.
• Contributor is new to the project; the change is small and well-scoped, but review depth on edge cases (partial failure, catch handler throwing, nested haystacks) is unverified from release notes alone.

Key questions

• Where is haystack state persisted in our configuration (cache driver vs. database), and does that store have a TTL or pruning job? Does the fix override an existing TTL?
• Does "do not delete on failure" also apply when the catch callback itself throws, or when the process is killed mid-flight?
• Is there a supported finish()/cleanup API we should call explicitly after handling a failure, or does state now persist indefinitely by design?
• Is this covered by a regression test upstream, and is the behaviour documented as intended semantics rather than incidental?
• What is the project's release cadence and versioning discipline — can we treat patch releases as safe to auto-merge?

Integration Approach

Stack fit

• Requires Laravel with a queue driver that supports delayed/sequenced dispatch (Redis, database, SQS). No new infrastructure needed for v2.2.0 itself.
• Pairs naturally with existing observability: failed haystack records become a first-class data source for dashboards and support tooling, rather than something reconstructed from logs.

Migration path

• Treat as a routine patch bump: update the constraint, run the suite, deploy. No code changes expected.
• Before rollout, inventory every place in our codebase that dispatches a haystack and confirm each has either (a) a catch handler that resolves the failure, or (b) an explicit operator runbook for inspecting and clearing retained state.
• If any internal tooling, alert, or dashboard currently infers failure from a missing haystack record, it must be inverted to key on a failure status field. This is the single most likely source of post-deploy confusion and should be found in advance, not after.

Compatibility

• Backwards compatible at the API level; behaviourally compatible for anyone who never inspected failure state. Flag as "safe to upgrade, review failure tooling" rather than "no-impact patch."
• Confirm the package's supported Laravel/PHP matrix against our oldest and newest services before bumping the floor anywhere.

Sequencing

1. Read the diff for PR #86 directly (release note is one line; do not upgrade on the note alone).
2. Confirm retention/pruning behaviour in a non-production environment, including a forced mid-chain failure and a throwing catch.
3. Update failure-inspection tooling and runbooks to read retained state.
4. Roll out to one non-critical consumer, monitor store growth for a defined window.
5. Roll out broadly; add a pruning or archival job if retention is unbounded.
6. Only then consider widening adoption to higher-value workflows.

Operational Impact

Maintenance

• Low ongoing cost: single-purpose package, small surface area, patch-level change. Ownership should sit with the team that owns the queue infrastructure, since failure state now lives in shared storage.
• Add a periodic review of open issues/PRs — a one-line release note for a semantic change suggests we should read diffs rather than trust changelogs for this dependency.

Support

• Net positive for support: failed haystacks are now inspectable after the fact, which shortens triage and enables self-service "what failed and where" answers.
• Requires a documented answer to "how do I clear a failed haystack?" Support will be asked this within the first week.

Scaling

• Retention is the scaling axis, not throughput. A persistent systemic failure (bad deploy, downstream outage) creates one retained record per attempt per haystack; at high volume this is a meaningful and sudden growth in the state store.
• Recommend a retention window and, if the package does not provide one, a scheduled prune keyed on failure timestamp — sized against observed failure rates rather than guesses.

Failure modes

• Retained-state accumulation: systemic failure plus no pruning degrades the cache/DB. Mitigate with retention.
• Silent success assumption: if alerting previously relied on disappearance of the record, failures may now go unnoticed. Verify alerting before rollout.
• Idempotency: if any workflow re-dispatches a haystack on failure, retained record identity must not cause duplicate execution or state collisions — worth a targeted test.
• Partial completion: mid-chain failure leaves earlier jobs committed; retained state makes this visible but does not make it reversible. Compensation logic remains the application's responsibility.

Ramp-up

• Minimal for engineers already fluent in Laravel queues — the mental model (jobs, chains, catch) maps directly. Estimate half a day for a developer to be productive, plus a short written runbook for on-call.
• The main learning is organisational, not technical: teams must internalise that failed haystacks persist, and that inspecting and clearing them is a normal, expected operation rather than an anomaly.