Realtime Job Batching Laravel Package

Getting Started

1. Install and publish.

composer require yogameleniawan/realtime-job-batching
php artisan vendor:publish --tag=realtime-job-batching-config   # verify tag in README

2. Confirm the transport is real. The package broadcasts through Pusher, so three things must all be true before you debug anything else:

• BROADCAST_DRIVER=pusher in .env (not log, not null)
• PUSHER_APP_ID, PUSHER_APP_KEY, PUSHER_APP_SECRET, PUSHER_APP_CLUSTER set
• php artisan config:clear (or re-run config:cache if you cache config in deploy)

3. Read the source first — this is a 24-star package with zero dependents. Budget 20 minutes in vendor/yogameleniawan/laravel-queue-realtime-progress/src before designing around it. You are looking for:

• the broadcast event class and its broadcastOn() channel naming
• whether it hooks into Laravel's native Bus::batch() or ships its own batch wrapper
• whether progress is broadcast per-job, or only at batch lifecycle boundaries (then/catch/finally)

That last question determines your entire UX design, so answer it before writing frontend code.

4. First use case — the one this package is actually good for: a user-triggered bulk operation (CSV import, bulk re-index, mass notification) where the request returns immediately with a batch ID and the UI shows a live progress bar. Do not start with high-throughput internal queues; see Gotchas.

$batch = Bus::batch($jobs)
    ->onConnection('database')
    ->onQueue('imports')
    ->allowFailures()
    ->name('import-'.$import->id)
    ->dispatch();

return response()->json(['batch_id' => $batch->id]);

5. Frontend: subscribe with Laravel Echo on whatever channel broadcastOn() returns, then render progress, processedJobs, failedJobs, finished.

Implementation Patterns

Store the batch ID, then hand it to the client. The batch object is returned synchronously from dispatch(), so persist $batch->id on your own model (imports.batch_id) before returning. The client needs an identifier to subscribe to before the first job finishes, and you'll want it later for reconciliation after a page reload.

Treat the broadcast as a cache invalidation, not the source of truth. The robust pattern is: event arrives → client either uses the payload directly (fast path) or hits GET /batches/{id} which reads $batch->progress() from the job_batches table (correct path). This survives dropped messages, reconnects, and closed laptops for free.

Per-job progress inside a job. For long single jobs, use the Batchable trait and emit your own granularity from the job body:

use Illuminate\Bus\Batchable;

class ImportChunk implements ShouldQueue
{
    use Batchable;

    public function handle(): void
    {
        foreach ($this->rows as $i => $row) {
            $this->process($row);

            if ($i % 50 === 0) {
                broadcast(new ChunkProgress($this->batch()->id, $i / count($this->rows)));
            }
        }
    }
}

Throttle by modulo, not per-row — see Gotchas.

Always broadcast a terminal state. The single most common bug in these dashboards: then broadcasts 100%, but catch/failure paths don't, and the UI hangs at 97%. Wire ->then(), ->catch(), and ->finally() to all emit an explicit status: finished|failed event. finally fires in every case and is the safest place for the terminal emit.

Channel scoping. One private channel per batch (batches.{batchId}) scales fine and keeps the auth callback trivial. A per-user channel is tempting but you end up fanning out every batch to every tab. If you go per-user, at minimum add the batch ID to the payload so the client can filter.

Authorize in routes/channels.php:

Broadcast::channel('batches.{batchId}', fn ($user, $batchId) =>
    Import::where('batch_id', $batchId)->where('user_id', $user->id)->exists()
);

Local development loop. QUEUE_CONNECTION=sync makes progress jump 0 → 100 instantly and teaches you nothing. Use database or redis plus a real php artisan queue:work, and keep php artisan pail running in a second pane so you see the broadcast events fire.

Integration tip: wrap dispatch in a small service class (BatchProgressService::dispatch($jobs, $user)) so the channel-naming and DB-persistence logic lives in one place. When you later swap the package out for your own event class, you change one file.

Gotchas and Tips

Pusher message quota is the real constraint. At 1 broadcast per job, a 10,000-job batch blows through the free tier's 200k messages/day in twenty runs, and you'll also hit per-second rate limits, causing dropped or delayed events under load. This package's sweet spot is tens to low hundreds of jobs. Above that, aggregate: broadcast every N jobs, on a Cache::lock-guarded interval (e.g. max 1 event per second per batch), or drop to Laravel's native polling on $batch->progress() and skip Pusher entirely.

Out-of-order delivery is normal. Two workers broadcasting concurrently means the client can receive progress: 60 before progress: 40. Never set(progress) blindly — take Math.max(current, incoming) so the bar never walks backwards.

Pusher payloads cap around 10KB. Keep broadcastWith() to scalars: batchId, progress, processed, failed, total, status. Don't serialize exception traces or per-item arrays into the event; they belong in logs and a REST fetch.

Progress is integer-floored. $batch->progress() returns int, so a 3-job batch reads 0 / 33 / 66 / 100. For small batches, compute your own float from processedJobs / totalJobs.

Nested batches are unsupported by Laravel itself, not just this package. If your job dispatches another batch, outer progress will read 100% while inner work is still running. Flatten, or add explicit counter jobs.

finally receives a freshly-loaded Batch model, so counts there can be one tick stale relative to the last per-job event. Emit the terminal event with counts you just read from the batch, and let the client reconcile via the REST endpoint rather than trusting an exact 100%.

Stuck-at-running is a real failure mode. If a worker is OOM-killed mid-job, the batch can sit in pending indefinitely with no terminal broadcast. Add a scheduled command that flags batches older than N hours with pending() > 0 and no recent updated_at change, and surface that as a failure in the UI.

Broadcasting runs inside the queue worker, so the worker process itself needs valid Pusher credentials and outbound HTTPS. A worker on a locked-down host silently fails to broadcast — Laravel logs the exception but your dashboard just stops updating with no client-side error. Watch storage/logs/laravel.log on the worker host, and check failed_jobs for broadcast-related exceptions.

Testing: Bus::fake() prevents batches from existing, so callbacks and broadcasts never fire — fine for asserting dispatch, useless for asserting progress. Event::fake() is what you want for the broadcast assertions. For a genuine end-to-end test, use QUEUE_CONNECTION=database and Bus::batch(...)->dispatch() in a RefreshDatabase test, then assert on the job_batches row.

Extension points to look for when the package falls short: most of these packages expose a configurable event class or channel prefix. If it doesn't, you can subclass the event, override broadcastOn(), and fire your own — you're not gaining much by forking. And since there are zero dependents upstream, expect to read src/ rather than find a Stack Overflow answer.