Laravel Sqs Sns Subscription Queue Laravel Package

Getting Started

Install and wire it up

composer require joblocal/laravel-sqs-sns-subscription-queue

The service provider is auto-discovered (Laravel 5.5+), so there is nothing to register. It hooks into Illuminate\Queue\QueueManager and adds a driver named sqs-sns alongside the built-in sqs driver.

Add a connection in config/queue.php:

'sqs-sns' => [
    'driver' => 'sqs-sns',
    'key'    => env('AWS_ACCESS_KEY_ID'),
    'secret' => env('AWS_SECRET_ACCESS_KEY'),
    'prefix' => env('SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/123456789012'),
    'queue'  => env('SQS_QUEUE', 'orders-events'),
    'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
    'suffix' => env('SQS_SUFFIX'),
    // SNS topic ARN this connection publishes to — confirm the exact
    // key name in SqsSnsConnector for the version you install:
    'sns' => [
        'topic' => env('SNS_TOPIC_ARN'),
    ],
],

Then set QUEUE_CONNECTION=sqs-sns (globally or per-environment).

AWS side setup (do this before writing any code)

1. Create an SNS topic.
2. Create an SQS queue.
3. Subscribe the queue to the topic — leave Raw Message Delivery OFF, otherwise you don't need this package at all.
4. Grant the topic sqs:SendMessage on the queue policy.

Where to look first

• src/LaravelSqsSnsSubscriptionQueueServiceProvider.php — tells you the exact driver string and which connector class is bound.
• src/SqsSnsConnector.php — shows the exact config keys (the SNS option name lives here).
• src/SqsSnsQueue.php — the actual behaviour: how push publishes to SNS and how pop unwraps the envelope. This is the file you will debug against. Read pop() and the unwrap method before trusting any behaviour documented anywhere else; the README is thin.

First use case

The canonical scenario is: an existing Laravel job class that already works on the sqs driver, dispatched with dispatch(new ProcessOrder($id)), consumed by php artisan queue:work. You should not need to change the job class at all. Prove the round trip works on a single job, on a real AWS topic/queue pair (LocalStack works too), before migrating anything else.

The one thing to internalise up front

This package only exists because SNS wraps your job payload in a notification envelope:

{
  "Type": "Notification",
  "MessageId": "...",
  "TopicArn": "...",
  "Message": "{\"uuid\":\"...\",\"displayName\":\"App\\\\Jobs\\\\ProcessOrder\",\"job\":\"Illuminate\\\\Queue\\\\CallQueuedHandler@call\",\"data\":{...}}",
  "MessageAttributes": { ... }
}

Vanilla Laravel's SqsQueue calls json_decode($body, true) and looks for a job key — it finds Type, MessageId, Message instead, and throws CallQueuedHandler / "job payload not found" style errors, or fails to resolve the class. This connector decodes the outer envelope, pulls out Message, and hands that to the normal job pipeline. Keeping this mental model makes every gotcha below obvious.

Implementation Patterns

Pattern 1 — Consume a topic owned by another service

The most common real use. Some other team publishes domain events to SNS; you own an SQS queue subscribed to it. Your jobs are ordinary classes implementing ShouldQueue; the constructor just needs to map from the SNS message body.

final class HandleOrderPlaced implements ShouldQueue
{
    public function __construct(public array $payload) {}

    public function handle(OrderProjector $projector): void
    {
        $projector->apply($this->payload);
    }
}

If a foreign publisher is producing messages that aren't Laravel job payloads at all, this package is the wrong tool — you want the raw SQS body, not Laravel's job deserialiser. Put a small custom consumer on the queue instead, or subscribe with the sqs driver plus Raw Message Delivery and read the body yourself.

Pattern 2 — Publish from Laravel, consume in Laravel

Fan-out to multiple services. Laravel publishes to the SNS topic; each subscriber's SQS queue gets its own copy. Keep the publishing connection and consuming connection as the same sqs-sns connection entry so the envelope format is symmetric. Do not mix an sqs publisher with an sqs-sns consumer unless you have verified the unwrap logic tolerates a bare payload.

Pattern 3 — Two connections side by side

Very common and recommended:

'sqs'     => [ ... ],   // direct queues, no SNS
'sqs-sns' => [ ... ],   // fan-out topic

Use onConnection('sqs') for internal fire-and-forget work, onConnection('sqs-sns') for domain events that other services care about. This limits your exposure to the package's rough edges to the fan-out path only.

Pattern 4 — Environment-split

'default' => env('QUEUE_CONNECTION', 'sync'),

Local and CI get sync (or database); staging/production get sqs-sns. This is the single highest-value integration tip: it means developers never need LocalStack/SNS emulation to run the test suite, and it keeps the SNS wiring exercised only where it matters.

Pattern 5 — Chained and batched jobs

Chaining (->chain([...])) and Bus::batch() rely on the queue pushing follow-up jobs after handling. Verify these explicitly on this driver before relying on them in production — chained jobs re-dispatch through the same connection and inherit all the push limitations noted below. Bus::batch() has extra requirements (it polls the job_batches table, and needs ->allowFailures() semantics wired correctly); test it early rather than discovering the problem under load.

Pattern 6 — Reading SNS metadata

If you need MessageAttributes (routing keys, tenant IDs, correlation IDs), check what the job wrapper exposes on the consumed side — SqsSnsQueue builds a job object around the original SQS message, so the SNS metadata should be reachable from $job in middleware or in handle() via Job-typed injection. If it isn't exposed cleanly, the pragmatic route is to put routing data in the job payload before publishing rather than relying on SNS attributes.

Integration tips

• Deploy the queue worker config change and the config/queue.php change together; a worker running the old driver against an SNS-wrapped queue will fail every job and burn through retries.
• Roll out on a new queue, not in place. Drain the old one, then switch the connection.
• Set failed_jobs up (php artisan queue:failed-table / migrate) before the first deploy. You will need it during the transition.
• Wire a CloudWatch alarm on ApproximateNumberOfMessagesVisible and on the NumberOfMessagesSent vs NumberOfMessagesDeleted delta for the queue — envelope bugs tend to manifest as messages that are received but never deleted.
• Put a DLQ (redrive policy) on the SQS queue. With fan-out you cannot "replay" by republishing to the topic without duplicating for every subscriber, so the DLQ is your only clean recovery path.

Gotchas and Tips

Raw Message Delivery is the escape hatch — evaluate it first. On the SNS subscription, set RawMessageDelivery = true and the SQS body becomes the bare Laravel payload; vanilla sqs driver then works and you can drop this package entirely. You lose SNS-level MessageAttributes and any filter-policy metadata on the delivered message. If you don't need those, do this instead. That is genuinely the right answer for a large fraction of teams.

onQueue() and delay() are the big behavioural landmines.

• SNS Publish has no DelaySeconds. dispatch(new Job)->delay(now()->addMinutes(5)) cannot be honoured by publishing to SNS. If the package accepts it silently you will get immediate execution; if it throws, you find out at runtime. Read SqsSnsQueue::push() and confirm, and add a test asserting the behaviour.
• The "queue name" for a fan-out topic is really a subscription, not a routing key. ->onQueue('high-priority') generally cannot redirect to a different SQS queue through a single SNS topic. Use SNS filter policies on MessageAttributes plus multiple subscribed queues if you need routing, and expect to bypass onQueue() for it.

FIFO needs verification. SNS FIFO topics require MessageGroupId (and optionally MessageDeduplicationId) on Publish. Check whether the connector forwards these. SQS FIFO queue names end in .fifo, and Laravel's stock SqsQueue detects that suffix to add MessageGroupId — with an SNS hop that logic no longer applies at the right layer.

SubscriptionConfirmation messages will poison the queue. If a subscription is created but not confirmed, or is confirmed out-of-band, SNS drops a {"Type":"SubscriptionConfirmation", ...} message into SQS. Your job pipeline will choke on it. Confirm subscriptions promptly, and consider a small filter (a middleware or a custom consumer) that deletes non-Notification message types instead of failing them into the DLQ.

Message size shrinks. SNS and SQS both cap at 256 KB, but the SNS envelope plus base64/attribute overhead eats into that. Jobs that squeaked under the limit on plain SQS can start failing at publish. If you carry large payloads, store the blob in S3 and pass a reference.

The receipt handle must survive the unwrap. Deleting and releasing a message requires the original SQS ReceiptHandle. If the queue class builds a fresh job object from the inner Message JSON instead of wrapping the original SQS message, then delete() / release() / retry semantics break — you'll see messages redelivered forever or "attempted to delete a message that is not in flight". Sanity-check this in SqsSnsQueue: the constructed job should hold the raw SQS message. Test by deliberately failing a job and confirming it retries with an incrementing attempts count and finally lands in failed_jobs.

after_commit / afterCommit. Laravel's queue config supports 'after_commit' => true. Confirm this passes through for a custom driver — dispatch-after-commit is a correctness issue for anything that reads the record it just created.

Local development is the friction point. Either use sync/database locally as described above, or run LocalStack (which supports SNS→SQS) or ElasticMQ for SQS only. Note that a bare SQS emulator will not wrap messages in an SNS envelope, so a locally-pushed job through the sqs-sns connection may arrive unwrapped and behave differently from production. Treat "works locally" as weak evidence for this driver.

Don't combine with Laravel Vapor. Vapor ships its own SQS/SNS handling and queue runtime; this connector and Vapor's will fight over the driver registration and the worker lifecycle.

Debugging: read the raw body first. When a job fails oddly, bypass the app entirely:

aws sqs receive-message --queue-url <url> --max-number-of-messages 1 \
  --message-attribute-names All --attribute-names All --visibility-timeout 0

Paste the Body into your log and inspect the real shape. Nine times out of ten the problem is obvious there: wrong Type, a Message that decodes to something other than a Laravel job payload, or a double-encoded string where you expected an object.

Test the unwrap in isolation. You can unit-test SqsSnsQueue by injecting a mocked Aws\Sqs\SqsClient and a fake SNS client, then calling pop() with a canned envelope body from a fixture file. That is far faster than integration testing against AWS and it's the only way you'll catch envelope-format regressions when AWS or the package changes something.

Version and maintenance risk is real. ~47 stars, last release 2023-09-28. Check composer.json for the supported illuminate/queue range against your Laravel version before committing, and check whether Aws\Sqs\SqsQueue internals changed (SqsQueue::pop() / getJobFromSqs() are the extension points and have shifted across Laravel majors). The class surface area here is small — SqsSnsConnector, SqsSnsQueue, and a provider — so if the package falls behind, vendoring and patching those three files is a realistic afternoon's work and probably a better bet than waiting for an update. Pin the version and add a note in your README about who owns that patch if it becomes necessary.