Laravel Haystack Laravel Package

Getting Started

Laravel Haystack lets you chain many queued jobs into a single "haystack" that runs sequentially, with shared data, delays between jobs, and pause/resume support — all persisted to the database so it survives restarts and doesn't balloon memory like a native Bus::chain of thousands of jobs would.

Minimal setup:

composer require sammyjo20/laravel-haystack
php artisan vendor:publish --tag="haystack-migrations"
php artisan migrate

This creates two tables you'll interact with indirectly: haystacks (the chain) and haystack_jobs (individual jobs + their order/status). There's also a published config/haystack.php controlling the database connection and model bindings — that's the first place to look if you need multi-tenant or separate-DB setups.

First use case — a simple sequential pipeline:

use Sammyjo20\LaravelHaystack\Facades\Haystack;

Haystack::build()
    ->addJob(new ProcessPodcast($podcast))
    ->addJob(new OptimizePodcast($podcast))
    ->addJob(new ReleasePodcast($podcast))
    ->dispatch();

Each job must use the Haystackable trait. That trait gives jobs access to $this->data (the shared payload) and lets them read/write across the chain.

Implementation Patterns

Shared data between jobs. Pass initial state with ->withData([...]), and read it in a job via $this->data['key']. Mutations made to $this->data persist forward to subsequent jobs, which makes this a good fit for multi-step workflows (imports, exports, ETL) where each step enriches a payload.

Delays between jobs. ->addJob(new Job, delay: 60) schedules a gap before a specific job, or ->withDelay(60) applies a default. Combined with pause/resume this is your rate-limit-throttled batch pattern (e.g. syncing thousands of records against a third-party API).

Lifecycle callbacks. ->then(fn () => ...), ->catch(fn () => ...), and ->finally(fn () => ...) run around the chain. Use these for notification, cleanup, or metrics rather than cramming logic into the last job. Callbacks can be closures or invokable classes.

Named haystacks + resume. ->withName('nightly-import') gives you a lookup handle so a controller/command can later find, pause, or resume the chain. Pausing is the escape hatch when a downstream service goes down mid-chain.

Composition and dispatch options. Haystack dispatch respects ->onConnection(...), ->onQueue(...), and standard queue knobs, so it slots in alongside your existing queue topology. Keep job classes small and idempotent — the chain itself doesn't give you exactly-once semantics.

Gotchas and Tips

• v2.2.0 behavior change: failed haystacks are no longer deleted. Prior to this release, a job failure tore down the haystack record. As of v2.2.0 the haystack (and its jobs) is retained on failure so you can inspect state and retry. Consequence: you now accumulate failed haystacks in the haystacks/haystack_jobs tables. Add a pruning routine or an ->catch() that cleans up, or your DB will grow without bound on flaky pipelines. This is worth a follow-up on your monitoring.
• Don't rely on in-memory state. Every job is a fresh deserialization. The only continuity is $this->data; anything else you need must be stored on the haystack or in your domain models.
• Serialization discipline. Whatever goes into ->withData() must be serializable. Don't stash closures, resources, or heavy Eloquent graphs — store IDs and re-fetch in the job (the SerializesModels trait handles model references, but raw data payloads don't get that treatment).
• Delays are per-job, not global. withDelay sets a default applied at add-time; it isn't a dynamic throttle. For adaptive backoff, pause and re-dispatch.
• Debugging failures. With the v2.2.0 retention change, failed haystacks stay queryable — inspect haystack_jobs to see which step failed and its payload. Check failed_jobs too, since the underlying job still fails the normal way.
• Config quirks. If you publish config on a multi-connection app, set the connection used by the Haystack models; the default database connection will surprise you in tenancy setups.
• Extension points. Models are swappable via config — useful for adding scopes or soft deletes on haystacks. The callbacks (then/catch/finally) are the cleanest hook for cross-cutting concerns rather than subclassing the Haystack itself.