Sqs Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require enqueue/sqs
   

   Ensure your Laravel project has the enqueue/amqp-laravel or enqueue/doctrine package for integration (SQS is a transport, not a full queue system).

2. Configuration: Add SQS transport to your Laravel queue configuration (config/queue.php):

   'connections' => [
       'sqs' => [
           'driver' => 'sqs',
           'host' => env('AWS_SQS_HOST', 'sqs.us-east-1.amazonaws.com'),
           'region' => env('AWS_REGION', 'us-east-1'),
           'credentials' => [
               'key' => env('AWS_ACCESS_KEY_ID'),
               'secret' => env('AWS_SECRET_ACCESS_KEY'),
           ],
           'queue' => env('SQS_QUEUE_URL', 'your-queue-url'),
           'options' => [
               'prefix' => env('SQS_PREFIX', ''),
               'suffix' => env('SQS_SUFFIX', ''),
           ],
       ],
   ],
   

3. First Use Case: Dispatch a job using SQS:

   use Illuminate\Support\Facades\Queue;
   
   Queue::connection('sqs')->push(new YourJobClass);
   

Key Files to Review

• config/queue.php – Laravel queue configuration.
• app/Console/Kernel.php – Queue worker setup.
• SQS Transport Docs – Official documentation.

Implementation Patterns

Workflows

1. Job Dispatching: Use SQS for async tasks (e.g., sending emails, processing uploads):

   Queue::connection('sqs')->push(
       new SendWelcomeEmail($user),
       'emails' // Optional custom queue name
   );
   

2. Worker Setup: Run the SQS worker via Artisan:

   php artisan queue:work sqs --queue=emails --sleep=3 --tries=3
   

   • --sleep: Delay between checks (seconds).
   • --tries: Max retries for failed jobs.

3. Batch Processing: Use batch() for grouped jobs:

   Queue::connection('sqs')->batch([])->push(new ProcessOrdersJob);
   

4. Delayed Jobs: Schedule jobs with delay():

   Queue::connection('sqs')->later(now()->addMinutes(10), new NotifyUserJob);
   

Integration Tips

• Laravel Events: Bind SQS to event dispatching:

  event(new UserRegistered($user))->onQueue('sqs', 'users');
  

• Retry Logic: Leverage SQS visibility timeout (default: 30s) for retries. Adjust in config/queue.php:

  'options' => [
      'visibility_timeout' => 60, // Seconds
  ],
  

• Monitoring: Use AWS CloudWatch or SQS metrics to track queue depth/latency.

Gotchas and Tips

Pitfalls

1. Credentials Management:

   • Never hardcode AWS keys. Use Laravel’s .env or AWS IAM roles (for EC2/ECS).
   • Rotate keys periodically via aws configure.

2. Visibility Timeouts:

   • Long-running jobs may trigger duplicate processing if the worker crashes before completing.
   • Fix: Extend visibility timeout or use Queue::later() for long tasks.

3. Queue Naming:

   • SQS queues are case-sensitive. Use consistent naming (e.g., snake_case).
   • Tip: Prefix queues with your app name to avoid collisions:

     'options' => ['prefix' => 'laravel_app_'],
     

4. Message Size Limits:

   • SQS max message size: 256KB (including headers).
   • Workaround: Store large payloads in S3 and reference the URL in the SQS message.

5. Worker Stuck on Messages:

   • If a worker hangs, manually change the message visibility via AWS Console or CLI:

     aws sqs change-message-visibility --queue-url YOUR_QUEUE_URL --receipt-handle RECEIPT_HANDLE --visibility-timeout 0
     

Debugging

• Enable Logging: Add to config/queue.php:

  'log' => env('QUEUE_LOG', true),
  'log_file' => storage_path('logs/queue.log'),
  

• Check Dead Letter Queues (DLQ): Configure DLQ in config/queue.php:

  'options' => [
      'dead_letter_queue' => 'https://sqs.us-east-1.amazonaws.com/1234567890/dlq',
  ],
  

  Failed jobs auto-route to DLQ after max retries.

Extension Points

1. Custom Middleware: Attach middleware to SQS jobs (e.g., logging, rate limiting):

   Queue::connection('sqs')->push(new YourJob)
       ->onQueue('high_priority')
       ->delay(10)
       ->middleware([\App\Middleware\LogJob::class]);
   

2. Extend Transport: Override SQS behavior by creating a custom transport class (extend Enqueue\Sqs\SqsContext).

3. Use Raw SQS API: For advanced use cases, inject the Aws\Sqs\SqsClient into a service:

   use Aws\Sqs\SqsClient;
   use Enqueue\Sqs\SqsContext;
   
   $sqs = new SqsClient([...]);
   $context = new SqsContext($sqs, $config);
   

4. Cross-Account Access: Use IAM roles or cross-account policies if accessing SQS across AWS accounts. Example policy:

   {
       "Version": "2012-10-17",
       "Statement": [{
           "Effect": "Allow",
           "Action": ["sqs:*"],
           "Resource": "arn:aws:sqs:us-east-1:1234567890:your-queue"
       }]
   }