Laravel Queue Rate Limit Laravel Package

Getting Started

Start by upgrading and confirming worker behaviour — this patch is a drop-in bugfix, so there is no config migration and no API surface change:

composer require <vendor>/<package>:^3.6.1
php artisan config:clear && php artisan queue:restart

Where to look first in the codebase:

1. app/Providers/AppServiceProvider.php (or a dedicated RateLimitServiceProvider) — every named limiter your jobs use via RateLimiter::for('...'). This is the source of truth for the windows the worker now waits on.
2. Your job middleware — search for RateLimited, RateLimitedWithRedis, or a custom release()-based throttle. These are what interact with the fixed behaviour.
3. Your worker invocations — app/Console/Kernel.php schedule entries, docker/entrypoint.sh, Kubernetes Job/CronJob manifests, and supervisor.conf, specifically anything passing --stop-when-empty.

Your first use case is the exact scenario that was broken: a one-shot worker draining a rate-limited queue.

php artisan queue:work redis --queue=exports --stop-when-empty --max-time=300

Pre-3.6.1 this could exit while released jobs were still waiting on a limiter window. Post-3.6.1 the worker blocks until the nearest window opens and continues. Verify on staging by dispatching more jobs than your limiter's perMinute allows and confirming the exit code is only returned after the last job completes.

Implementation Patterns

One-shot drain for batch work. The canonical pattern the fix enables — dispatch N jobs to a throttled queue, then run a single worker that exits when genuinely done:

// Dispatch
ExportReport::dispatch($report)->onQueue('exports');

// Then, out-of-band (scheduler, CI step, controller):
Artisan::call('queue:work', [
    'connection' => 'redis',
    '--queue' => 'exports',
    '--stop-when-empty' => true,
    '--max-time' => 300,   // always pair; see Gotchas
    '--tries' => 3,
]);

Bound every waiting worker. Because a rate-limited worker now waits rather than exiting, treat --max-time as mandatory whenever --stop-when-empty is combined with throttling. --max-time gives you a hard ceiling; your cron interval gives you the retry cadence.

Scheduled drain instead of a long-lived worker. For bursty, throttled workloads, prefer short scheduled drains over an always-on daemon — the exit-when-empty semantics give you natural backpressure and cheap idle cost:

$schedule->command('queue:work redis --queue=exports --stop-when-empty --max-time=240')
         ->everyFiveMinutes()
         ->withoutOverlapping();

withoutOverlapping() is not optional here: a waiting worker plus a 5-minute cadence is how you get two workers contending for the same limiter slots.

Respect multi-queue ordering. The release explicitly preserves the existing priority order for --queue=high,default,low. Keep high-priority queues first; the worker still drains high fully before touching low, and the rate-limit wait only applies to the queue it is currently blocked on. If a throttled queue sits ahead of an unthrottled one, the unthrottled work is starved while the worker waits — order accordingly.

Choose your limiter middleware deliberately.

public function middleware(): array
{
    return [new RateLimited('exports')];
}

RateLimited releases the job back with the limiter's own availableIn() delay. If you have a custom middleware doing throttling, make it release($delay) — never delete() — otherwise the job is silently dropped and the fix cannot help you.

Pair with --max-jobs for predictable teardown. --max-time bounds wall clock, --max-jobs bounds throughput. Together with --stop-when-empty you get a worker that terminates on whichever comes first, which is what you want in a Kubernetes Job (exit 0 = success, no restart loop).

Test with a real queue, not Queue::fake(). The fake never exercises the limiter/worker interaction. A regression test that actually catches this class of bug needs a real connection:

RateLimiter::for('exports', fn ($job) => Limit::perMinute(1));

ExportReport::dispatch($a)->onQueue('exports');
ExportReport::dispatch($b)->onQueue('exports');

$this->artisan('queue:work', [
    'connection' => 'redis',
    '--queue' => 'exports',
    '--stop-when-empty' => true,
])->assertExitCode(0);

$this->assertSame(0, Queue::size('exports'));

Gotchas and Tips

The waiting worker can hang. The fix means "wait for the nearest rate-limit window," which is correct but unbounded if your limiter window is long relative to your cron interval. A limiter with Limit::perHour(10) can park a worker for six minutes. Always set --max-time below your scheduler interval, and treat a --max-time exit as incomplete work, not failure — the next scheduled run picks up the remainder.

--stop-when-empty is not what Horizon uses. If you run Horizon, this fix does not apply to you: Horizon supervises long-lived workers and has no exit-when-empty mode. Don't add the flag to Horizon config and expect different behaviour — use Horizon's balance and maxProcessTime settings instead, or run a separate non-Horizon worker for one-shot drains.

Exit codes deserve monitoring. --stop-when-empty exits 0 both when the queue drained and when --max-time cut it short in some versions. Don't alert on exit code alone; check the queue size after the run (via queue:monitor or a Queue::size() probe) and alert on a non-decreasing backlog across consecutive runs.

size() on SQS is approximate. --stop-when-empty leans on the queue's size(). On SQS this is eventually consistent and can report zero while in-flight messages exist. If you're on SQS with throttled jobs, prefer --max-time as the primary terminator and treat the empty check as a fast path only.

Rate-limit keys are your responsibility. RateLimiter::for('exports', fn ($job) => Limit::perMinute(100)->by($job->tenantId)) gives per-tenant windows; omitting ->by() gives you a global window that one noisy tenant can monopolise. Note the interaction: per-tenant keys mean one tenant's exhausted window blocks the worker for all tenants on that queue. Consider a queue-per-tier split before you go per-tenant with the limiter.

Debugging a worker that won't exit. Run with -v first, then instrument:

php artisan queue:work redis --queue=exports --stop-when-empty -v
php artisan tinker --execute="dump(RateLimiter::availableIn('exports'));"

If availableIn() returns a large value and the queue is non-empty, the worker is behaving as designed. If the queue is non-empty and the worker exits immediately, confirm you actually deployed 3.6.1 (composer show <vendor>/<package>) and that queue:restart was issued — long-lived workers keep old code in memory.

Extension point: custom limiters must return sane delays. Any custom RateLimited-style middleware you write should call $limiter->availableIn($key) and release() that exact value. Returning 0 or null produces a hot retry loop; returning a fixed large delay masks tuning problems and makes the new wait behaviour look like a hang.

Don't rely on this fix for correctness of your throttling. It fixes worker lifetime, not delivery guarantees. Keep --tries, retry_after, and failed-job handling configured — a job that keeps hitting an exhausted limiter still burns attempts unless you use RateLimited's release semantics (which do not increment attempts) rather than an exception-throwing throttle.