Laravel Queue Raw Sqs Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require pawprintdigital/laravel-queue-raw-sqs
   

   Publish the config (if needed):

   php artisan vendor:publish --provider="PawprintDigital\LaravelQueueRawSqs\RawSqsServiceProvider"
   

2. Configure Queue Connection Add a new connection in config/queue.php:

   'connections' => [
       'raw-sqs' => [
           'driver' => 'raw-sqs',
           'key' => env('AWS_ACCESS_KEY_ID'),
           'secret' => env('AWS_SECRET_ACCESS_KEY'),
           'region' => env('AWS_DEFAULT_REGION'),
           'queue' => env('RAW_SQS_QUEUE_URL'),
           'visibility_timeout' => 60,
       ],
   ],
   

3. First Use Case: Dispatch a Raw SQS Job

   use PawprintDigital\LaravelQueueRawSqs\Jobs\RawSqsJob;
   
   RawSqsJob::dispatch('my-queue-name', [
       'key' => 'value',
       'data' => ['nested' => 'payload']
   ]);
   

4. Consuming Raw Messages Register a listener in app/Providers/EventServiceProvider.php:

   protected $listen = [
       'PawprintDigital\LaravelQueueRawSqs\Events\RawSqsMessageReceived' => [
           'App\Listeners\HandleRawSqsMessage',
       ],
   ];
   

Implementation Patterns

Workflow: Processing Raw SQS Messages

1. Dispatching Messages Use RawSqsJob for structured payloads:

   RawSqsJob::dispatch('notifications', [
       'event' => 'user.created',
       'user_id' => 123,
   ]);
   

2. Handling Incoming Messages Implement a listener for RawSqsMessageReceived:

   public function handle(RawSqsMessageReceived $event) {
       $message = $event->message;
       $queue = $event->queueName;
   
       if ($queue === 'notifications') {
           $this->processNotification($message);
       }
   }
   

3. Batch Processing For high-volume queues, use RawSqsJob::dispatchMany():

   RawSqsJob::dispatchMany('batch-jobs', [
       ['id' => 1, 'type' => 'process'],
       ['id' => 2, 'type' => 'archive'],
   ]);
   

4. Integration with Existing Queues Combine with Laravel’s default queues:

   // Dispatch to SQS via raw-sqs connection
   dispatch(new ProcessOrder)->onQueue('raw-sqs');
   
   // Dispatch to default queue
   dispatch(new SendEmail)->onQueue('default');
   

Patterns for Extensibility

• Custom Message Handlers Override RawSqsMessageReceived logic in a service provider:

  public function boot() {
      RawSqsMessageReceived::extend(function ($message, $queue) {
          return new CustomMessageHandler($message, $queue);
      });
  }
  

• Middleware for Raw SQS Apply middleware to raw SQS jobs:

  RawSqsJob::dispatch('audit-log', $data)
      ->onConnection('raw-sqs')
      ->after(function ($job) {
          Log::info('Raw SQS job dispatched', $job->payload);
      });
  

Gotchas and Tips

Common Pitfalls

1. Visibility Timeout

   • Default is 60 seconds. Adjust in config if messages are processed longer:

     'visibility_timeout' => 300, // 5 minutes
     

   • Tip: Monitor VisibilityTimeout errors in CloudWatch for stuck messages.

2. Message Size Limits

   • SQS enforces a 256KB payload limit. For larger data, use S3 + SQS (store URL in SQS).
   • Tip: Serialize payloads efficiently (e.g., json_encode with JSON_THROW_ON_ERROR).

3. Duplicate Processing

   • SQS may redeliver messages. Use idempotency keys or deduplication logic:

     if (!ProcessedMessage::exists($message['id'])) {
         ProcessedMessage::create(['id' => $message['id']]);
         // Process message
     }
     

4. Connection Configuration

   • Ensure AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION are set in .env.
   • Tip: Use IAM roles for production deployments (avoid hardcoding keys).

Debugging Tips

• Log Raw Messages Add a listener to log incoming messages:

  public function handle(RawSqsMessageReceived $event) {
      \Log::debug('Raw SQS message', [
          'queue' => $event->queueName,
          'message' => $event->message,
      ]);
  }
  

• Check SQS Metrics Monitor ApproximateNumberOfMessagesVisible in AWS Console to debug queue backlogs.

• Test Locally Use localstack to mock SQS:

  docker run -it -p 4566:4566 localstack/localstack
  

  Update config:

  'url' => 'http://localhost:4566/000000000000/my-queue',
  

Extension Points

1. Custom Queue Naming Override queue name resolution:

   RawSqsJob::dispatchWithQueueName('custom-prefix-' . $queue, $payload);
   

2. Message Transformation Modify payloads before dispatch:

   RawSqsJob::dispatch('transformed', $data)
       ->transform(function ($payload) {
           return array_merge($payload, ['timestamp' => now()->toIso8601String()]);
       });
   

3. Error Handling Catch SQS-specific exceptions:

   try {
       RawSqsJob::dispatch('critical', $data);
   } catch (\Aws\Sqs\Exception\SqsException $e) {
       \Log::error('SQS dispatch failed', ['error' => $e->getMessage()]);
       // Retry or fallback logic
   }
   

4. Delayed Messages Use SQS delay queues (configure via AWS Console) or implement a wrapper:

   RawSqsJob::dispatch('delayed', $data)->delay(seconds: 300);