Queue Laravel Package

Getting Started

What you're actually installing

tarantool/queue (repo tarantool-php/queue) is a PHP client library for Tarantool's queue engine, not a Laravel queue driver. There is no queue:work tarantool driver, no Tarantool\Queue\ShouldQueue, and no service provider shipped with it. Day-to-day, a Laravel developer uses it one of two ways:

1. Directly — as a low-level FIFO/TTL/buried-task API inside an artisan command or a thin wrapper service (the common case).
2. As the transport behind a small custom Illuminate\Contracts\Queue\Queue driver so the normal dispatch() / queue:work workflow keeps working.

⚠️ If the previous version of this assessment told you to set QUEUE_CONNECTION=tarantool and run php artisan queue:work tarantool out of the box — that is not provided by this package. You must write the bridge (shown below).

Minimal setup

Requirements (changed in v0.11.0): PHP 8.2+, tarantool/client ^0.11, and a Tarantool instance with the queue module started.

composer require tarantool/queue:^0.11

Tubes must exist server-side before the client can touch them (the client does not create them):

-- init.lua / Tarantool console
queue.start()
queue.create_tube('default', 'fifottl', { temporary = true, ttl = 3600 })
queue.create_tube('high',    'fifottl', { temporary = true, ttl = 3600 })

Config + container binding:

// config/tarantool.php
return [
    'dsn'      => env('TARANTOOL_DSN', 'tcp://127.0.0.1:3301'),
    'user'     => env('TARANTOOL_USER', 'guest'),
    'password' => env('TARANTOOL_PASSWORD'),
    'queue'    => env('TARANTOOL_QUEUE', 'default'),
];

// app/Providers/AppServiceProvider.php
use Tarantool\Client\Client;
use Tarantool\Queue\Queue;

public function register(): void
{
    $this->app->singleton(Client::class, fn () => new Client(
        config('tarantool.dsn'),
        [
            'user'     => config('tarantool.user'),
            'password' => config('tarantool.password'),
        ],
    ));

    $this->app->bind('tarantool.queue', fn ($app) => new Queue(
        $app->make(Client::class),
        config('tarantool.queue'),
    ));
}

First use case — a producer/consumer pair

// Producer (controller, listener, command)
use Tarantool\Queue\Queue;

public function store(Request $request, Queue $queue)
{
    $task = $queue->put(['payment_id' => $request->integer('payment_id')]);

    if ($task === null) {          // NEW in v0.11.0: bounded tube is full
        return response()->json(['error' => 'queue full'], 503);
    }

    return response()->json(['task_id' => $task->getId()]);
}

// Consumer (artisan command run by supervisor)
$task = $queue->take(1);           // 1s blocking-ish wait; null when nothing available

if ($task !== null) {
    try {
        ProcessPayment::dispatchSync($task->getData()['payment_id']);
        $queue->ack($task->getId());
    } catch (\Throwable $e) {
        report($e);
        $queue->release($task->getId(), ['delay' => 30]);
    }
}

Where to look first

• src/Queue.php — the whole public surface: put, take, ack, release, bury, kick, peek, delete, stats. This file is the documentation.
• src/Task.php — getId(), getData(), getState(), getTtl(), getPrio(), created/updated timestamps.
• README "typed queue APIs" and "subqueues" sections — the v0.11.0 additions. Read these before writing a wrapper; they may replace hand-rolled DTO mapping.
• Tarantool queue module Lua docs (queue.start, queue.create_tube, queue.statistics) — tube creation and fifo / fifottl / limfifottl / utube / utubettl type choice happens there, not in PHP.

Implementation Patterns

1. Wrapping the client in a Laravel service (recommended baseline)

Don't inject Tarantool\Queue\Queue into controllers and jobs. Hide it behind one app-level class so you can fake it, swap it, and add telemetry:

namespace App\Support;

use Tarantool\Queue\Queue as Tube;

class TarantoolQueue
{
    public function __construct(private Tube $tube) {}

    public function push(array $payload, array $opts = []): ?string
    {
        return $this->tube->put($payload, $opts)?->getId();
    }

    public function reserve(float $timeout = 1.0): ?array
    {
        $task = $this->tube->take($timeout);

        return $task ? ['id' => $task->getId(), 'data' => $task->getData()] : null;
    }

    public function ack(int $id): void    { $this->tube->ack($id); }
    public function retry(int $id, int $delay): void { $this->tube->release($id, ['delay' => $delay]); }
    public function bury(int $id): void   { $this->tube->bury($id); }
}

2. Implementing a real Laravel queue driver (so dispatch() and queue:work work)

Register a driver that maps Laravel's contract onto the tube:

// app/Providers/QueueServiceProvider.php
use Illuminate\Support\Facades\Queue as QueueManager;

QueueManager::extend('tarantool', fn ($app) => new \App\Queue\TarantoolConnector(
    $app->make(\Tarantool\Queue\Queue::class),
    $app['db'],   // for the failed-jobs store
));

// config/queue.php
'connections' => [
    'tarantool' => ['driver' => 'tarantool', 'queue' => 'default', 'retry_after' => 90],
],

The connector implements Illuminate\Contracts\Queue\Queue:

And the job wrapper implements Illuminate\Contracts\Queue\Job:

public function delete()            { $this->tube->ack($this->task->getId()); }
public function release($delay = 0){ $this->tube->release($this->task->getId(), ['delay' => $delay]); }
public function attempts()          { return $this->task->getData()['attempts'] ?? 1; } // track your own counter
public function getRawBody()        { return $this->raw; }

Note: Tarantool has no attempts column and no failed_jobs table — you carry an attempt counter in the payload and record failures in Laravel's failed_jobs yourself.

3. TTL tubes and delayed work

Use fifottl/utubettl tubes to get Laravel-like delay and expiry for free:

$queue->put($payload, ['ttl' => 600, 'ttl_interval' => 30]);   // ready after delay, expires after 10 min
$queue->release($id, ['delay' => 60]);                          // backoff / retry

This is the main reason to reach for the package instead of a database queue: delayed and expiring work without a scheduler sweep.

4. Subqueues and subqueuettl (new in v0.11.0)

Scoped subqueues let you carve priority/per-tenant lanes out of one tube without creating dozens of tubes:

// Producer: named subqueue with its own TTL
$queue->put($payload, [
    'subqueue'    => 'tenant:42',
    'subqueuettl' => 300,     // subqueue lane expires; tasks don't accumulate forever
]);

// Scoped handle (v0.11.0): repeat the subqueue context once, not per call
$tenantQueue = $queue->subqueue('tenant:42');   // consult README for the exact factory name
$tenantQueue->put($payload, ['ttl' => 600]);
$task = $tenantQueue->take(1);

Pattern: use subqueues for short-lived partitioning (per-tenant fairness, per-request batch lanes) and real tubes for durable, long-lived routing (high, default, low). Pair with subqueuettl so dynamic names evaporate instead of leaking.

5. Typed queue APIs (new in v0.11.0)

The typed API gives you task/payload objects instead of mixed msgpack arrays. This is the natural seam for DTO-based jobs:

final readonly class PaymentPayload
{
    public function __construct(public int $paymentId, public int $attempt = 1) {}
}

$queue->putTyped(new PaymentPayload($id));   // check README for exact method names/signatures

Use it when you have a stable payload contract; keep the untyped put()/take() for legacy or polymorphic payloads. Don't mix both on the same tube without a discriminator field.

6. Consuming from Laravel

• Preferred: the custom driver above, then php artisan queue:work tarantool --sleep=1 --tries=3 --timeout=120 under Supervisor. Note --sleep governs polling between empty pop() calls; --timeout kills workers on long handlers.
• Direct workers: a dedicated artisan command with while (true) { take(); ... }, $this->handleSignal()-style stop handling, and one Tarantool client connection per worker process.
• Batching: Tarantool's queue has no native batch take — loop take() with timeout = 0 to drain non-blockingly.

7. Testing

// Unit: fake your own wrapper, not the package
$this->mock(TarantoolQueue::class, function ($m) {
    $m->shouldReceive('push')->once()->andReturn('42');
});

// Integration: real Tarantool in CI
docker run --rm -p 3301:3301 -v $(pwd)/tests/tarantool:/opt/tarantool tarantool/tarantool

Keep Queue::fake() for the Laravel side — it intercepts dispatch(), not the Tarantool client.

Gotchas and Tips

1. v0.11.0 upgrade checklist

• PHP 8.2+ is now required. Laravel 10 apps still on PHP 8.1 must either stay on tarantool/queue v0.10 or upgrade PHP first. Verify with composer why-not tarantool/queue 0.11.
• tarantool/client ^0.11 is required. If you pin an older client (or a fork), Composer will refuse the upgrade. Run composer update tarantool/client tarantool/queue --with-dependencies and re-test connection options (user/password/timeouts), since client option handling is what changed.
• Queue::put() can now return null. This is the one breaking behaviour change. Any code doing $queue->put($data)->getId() will fatal on a bounded tube (limfifottl, or a tube created with capacity). Audit every put() call site; add ?->getId() / explicit null handling. Decide the app-level contract: throw QueueFullException, retry with backoff, or shed load with a 503. Also check the typed API's return nullability — assume it can be null too.
• No CI-facing changes for you: the tarantoolctl removal and Actions bumps are internal.
• Everything else is additive — typed APIs and subqueues don't break existing put/take/ack code paths.

2. Tubes are created in Lua, not PHP

NoSuchTube-style errors on first run almost always mean queue.start() / queue.create_tube() wasn't executed, or temporary = true tubes vanished after a Tarantool restart. Ship the Lua init as a migration-like artifact (database/tarantool/init.lua) and run it in your container entrypoint.

3. Subqueue names are a leak vector

Dynamically named subqueues (tenant:{id}, batch:{uuid}, user:{id}:{date}) are the entire point of subqueuettl — but only if you actually set it. Rules:

• Always pass subqueuettl with a dynamic subqueue. Omitted TTL = permanent lane.
• Bound the cardinality. Never interpolate unbounded values (raw timestamps, UUIDs per request) — you'll create thousands of lanes and blow up queue.statistics() and memory. Derive names from a small hash bucket: tenant:bucket:' . ($id % 64).
• Observe lane growth via $queue->stats() in a scheduled command; alert when lane count exceeds a threshold.

4. No attempts, no failed_jobs, no retry command

The package gives you bury/kick/peek/delete; Laravel's queue:retry and queue:flush don't apply. Build the missing pieces:

• Carry attempts and maxTries in the payload; increment on take().
• On exceeding maxTries → bury() and write a row to Laravel's failed_jobs (or your own table) so queue:failed works.
• Dead-letter inspection: $queue->peek($id) in PHP, or queue.tube.<name>:peek(id) / queue.statistics() in the Tarantool console.

5. Payload serialization

Payloads cross the wire as msgpack via tarantool/client:

• Send arrays and scalars, not Eloquent models or objects with closures. Serialize models yourself (['model' => $class, 'id' => $id]) and re-fetch in the worker.
• Binary strings, floats and null keys round-trip fine, but associative arrays with integer-like string keys can come back as lists — normalise on read.
• $task->getData() returns mixed; validate with a DTO (or use the new typed API) rather than array-accessing blindly.

6. Delivery is at-least-once — make handlers idempotent

A take() without ack() (worker crash, timeout, killed process) returns the task to the tube after the tube's TTL/touch window. There is no exactly-once guarantee. Guard with Cache::lock("payment:{$id}") or a unique constraint, and ack after side effects complete.

7. Worker and connection pitfalls

• Don't share one Client across concurrently forked workers. Each Supervisor process should build its own connection (keep the binding singleton per-process, which Laravel does anyway — just don't preload/fork after connecting).
• take() with a large timeout blocks the connection. Long polls plus a slow handler serialise badly; use short waits with --sleep in the worker loop instead.
• Long handlers: --timeout=120 plus pcntl handling; on timeout the worker dies without acking and the task is redelivered — that's your retry mechanism, not a bug.
• Memory: --memory=128 restarts workers before msgpack buffers / client state grow.

8. Debugging

$queue->stats();                 // ready/taken/buried/done counters, per tube
$queue->peek($taskId);           // inspect a stuck task

-- Tarantool console
queue.statistics()               // global, includes subqueues
queue.tube.default:peek(task_id)
box.space.<tube_name>:select()

Log task_id in every dispatch and every ack/release — with Tarantool, the task id is your only correlation handle. Catch and log NoSuchTaskException separately from connection errors: the first usually means a double-ack or an expired TTL, the second means DSN/credentials.

9. Extension points

• Decorate, don't subclass, your wrapper: add metrics, tracing, and attempts bookkeeping in one place.
• Laravel events: fire your own TarantoolJobProcessed / TarantoolJobBuried events from the wrapper (there are no package-level Laravel events).
• Job middleware: if you go the custom-driver route, standard middleware() on job classes works unchanged; if you consume directly, implement middleware in your command's handler pipeline.
• Lua-side processing: for hot paths you can push a task and have Tarantool process it natively (process_payment.lua, invoked via queue.tube.<name>:take in Lua), keeping PHP out of the loop entirely — but then Laravel's dispatch() lifecycle no longer applies.
• Custom driver: Queue::extend('tarantool', ...) in a service provider is the sanctioned hook; the connector is where driver-level config (retry_after, block_for, queue) is interpreted.