Laravel Chunkable Jobs Laravel Package

Getting Started

• Install with composer require sammyjo20/laravel-chunkable-jobs. As of v1.1.0 the package supports Laravel 9–11 and PHP 8.3, so you can adopt it on a modern stack without pinning older framework versions.
• There is no service provider config to publish for normal use — the package is opt-in per job class. Read src/ChunkableJob.php (the contract) and the README examples first; that is the whole surface area.
• The first use case to reach for: a job that currently loops over a large collection (foreach ($users as $user) { ... }) inside a single queue worker. That is the classic timeout/memory killer. Convert it to a chunkable job, set a chunk size, and each chunk becomes its own queued job that re-dispatches the next one.
• Typical first conversion:

class SyncContacts implements ShouldQueue, ChunkableJob
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(public Collection $contacts) {}

    public function getChunkSize(): int
    {
        return 250;
    }

    public function handle(): void
    {
        foreach ($this->contacts as $contact) {
            // only ever 250 items in $this->contacts
        }
    }
}

Then dispatch with the full collection and let the package slice it: SyncContacts::dispatch($contacts).

Implementation Patterns

• Chunk size is the main tuning knob. Pick based on the slowest external call in the loop, not on item count. 100–500 for DB/API work, 1000+ for cheap in-memory transforms. Keep each chunk comfortably under your queue worker timeout (--timeout) and Horizon's timeout.
• Queue routing per chunk. Because each chunk is a separate dispatched job, you can push the whole run onto a dedicated queue or connection (e.g. imports on a low-concurrency supervisor) and let the chunking do the load-shedding for you.
• Lifecycle hooks. The dispatch builder exposes per-chunk callbacks (progress/onChunk and failure hooks — confirm exact names in the README for your version). Use them to write progress to cache or a JobRun model so the UI can show "1,200 / 40,000 processed" rather than a spinner.
• Pair with Bus::batch() for fan-out, not fan-in. If you have 20 independent large imports, batch the imports; use chunkable jobs within each import. Don't nest chunkable jobs inside an already-chunked loop.
• Keep chunks idempotent. A retried chunk re-runs the same slice. Prefer updateOrCreate, upserts, and "already done" guards over blind inserts. Store a cursor/last_id on a run record if you need resumability.
• Testing. Queue::fake() still works — assert the first job was pushed with the correct slice, then call handle() directly with a small collection to test the per-chunk logic. Assert getChunkSize() explicitly in a unit test so a refactor doesn't silently change throughput.
• Model serialization. SerializesModels re-fetches models on each chunk; if items may be deleted mid-run, pass IDs or plain DTOs instead of Eloquent models to avoid ModelNotFoundException partway through.

Gotchas and Tips

• No automatic checkpointing. If the queue dies mid-run (worker killed, Redis flush), the remaining chunks are gone. For long imports, persist a run record and provide a manual "resume from offset" path — or dispatch the next chunk yourself from a finally block so failures are visible.
• Ordering is not guaranteed on multi-worker queues. Chunks can complete out of order; never rely on chunk N finishing before N+1 when the work has dependencies.
• Retries multiply work. $tries = 3 on a chunkable job means a failing chunk retries 3 times, not the whole run — usually what you want, but it also means a permanently failing chunk stops the chain silently. Always implement failed() and alert/log there.
• Memory doesn't disappear, it moves. Chunking fixes a single worker timing out, but the dispatching request still holds the full collection. For very large sets, chunk from a query (->chunkById()) and dispatch one chunkable job per slice, or the originating web request becomes the new bottleneck.
• Unique jobs / ShouldBeUnique will interfere: consecutive chunks look identical to the unique lock and get dropped. Avoid uniqueness on the chunkable job class; use a run ID in the payload instead.
• Upgrade note: v1.1.0 is purely an additive compatibility release (Laravel 11 + PHP 8.3, PR #11). No breaking changes, no deprecations, no config migration from v1.0.0 — safe to bump in composer.json. If you're on PHP 8.3 + Laravel 11, this is the minimum version you should be on, since v1.0.0 predates that matrix.
• Extension points are thin by design. If you need custom behaviour, subclass the job and override the chunk-size or lifecycle methods rather than forking the package; the surface is small enough that a subclass is cheaper to maintain.