Aws Queue Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require beyerz/aws-queue-bundle
   

   Ensure beyerz/aws-sdk-bundle (dependency) is also installed.

2. Register Bundle Add to config/bundles.php:

   Beyerz\AwsQueueBundle\BeyerzAwsQueueBundle::class => ['all' => true],
   

3. Configure AWS Credentials Use beyerz/aws-sdk-bundle to configure AWS SDK (e.g., via config/packages/beyerz_aws.yaml):

   beyerz_aws:
       region: 'us-east-1'
       credentials:
           key: '%env(AWS_ACCESS_KEY_ID)%'
           secret: '%env(AWS_SECRET_ACCESS_KEY)%'
   

4. Define a Queue Create a Producer service to publish messages:

   # config/services.yaml
   services:
       App\Service\MyQueueProducer:
           tags: ['aws_queue.producer']
           arguments:
               $queueName: 'my_queue'
               $region: 'us-east-1'
   

5. First Use Case: Publish a Message Inject the producer into a controller or command:

   use Beyerz\AwsQueueBundle\Producer\ProducerInterface;
   
   class MyController extends AbstractController
   {
       public function __construct(private ProducerInterface $producer) {}
   
       public function publish()
       {
           $this->producer->send('Hello, AWS SQS!');
       }
   }
   

Implementation Patterns

Core Workflows

1. Producer-Consumer Pairing

   • Producers publish messages to SQS queues.
   • Consumers pull messages from SQS and process them.
   • Example consumer service:

     services:
         App\Service\MyQueueConsumer:
             tags: ['aws_queue.consumer']
             arguments:
                 $queueName: 'my_queue'
                 $region: 'us-east-1'
                 $handler: '@App\Service\MessageHandler'
     

2. Message Handling

   • Consumers delegate processing to a handler (e.g., a service with a handle() method):

     class MessageHandler
     {
         public function handle(string $message): void
         {
             // Process $message (e.g., save to DB, trigger events)
         }
     }
     

3. Error Handling

   • Use onError() in consumers to log or retry failed messages:

     class MyQueueConsumer
     {
         public function onError(string $message, \Exception $e): void
         {
             // Log or notify (e.g., via Monolog)
             throw new \RuntimeException('Failed to process: ' . $e->getMessage());
         }
     }
     

4. Batch Processing

   • Configure consumers to fetch messages in batches (default: 1):

     arguments:
         $batchSize: 10  # Fetch 10 messages per poll
     

Integration Tips

1. Symfony Events Trigger events when messages are published/consumed:

   // In Producer:
   $this->producer->send($message, ['event' => 'message.published']);
   

2. Dependency Injection

   • Prefer constructor injection for producers/consumers:

     public function __construct(private ProducerInterface $producer) {}
     

3. Environment-Specific Queues Use %env(QUEUE_NAME)% in services.yaml for dynamic queue names.

4. Testing

   • Mock ProducerInterface and ConsumerInterface in PHPUnit:

     $this->mockProducer->expects($this->once())->method('send')->with('test');
     

5. Dead-Letter Queues (DLQ) Configure DLQs in AWS and route failed messages via consumer’s onError().

Gotchas and Tips

Pitfalls

1. AWS Credentials

   • Ensure beyerz/aws-sdk-bundle is properly configured. Missing credentials will throw Aws\Sqs\Exception\SqsException.
   • Fix: Verify .env or config/packages/beyerz_aws.yaml.

2. Queue Permissions

   • SQS/SNS policies must allow the IAM role to send/receive messages.
   • Fix: Attach AmazonSQSFullAccess (temporarily for testing) or custom policies.

3. Consumer Visibility Timeout

   • Long-running consumers may cause messages to reappear if the timeout (default: 30s) expires.
   • Fix: Increase timeout in AWS SQS console or handle messages quickly.

4. Message Size Limits

   • SQS messages are capped at 256KB. Large payloads require base64 encoding or S3 storage.
   • Fix: Use json_encode() for small data or store large data in S3 and reference the URL.

5. Consumer Overlapping

   • Multiple consumers on the same queue may process the same message if not managed.
   • Fix: Use FIFO queues or implement idempotent handlers.

Debugging

1. Enable AWS SDK Debugging Add to config/packages/beyerz_aws.yaml:

   beyerz_aws:
       debug: true
   

   Logs will appear in var/log/dev.log.

2. Check Queue ARN

   • Ensure $queueName in services matches the SQS queue ARN (e.g., my_queue.fifo for FIFO).
   • Fix: Use aws sqs list-queues to verify queue names.

3. Consumer Stuck on Polling

   • Consumers may hang if no messages are available.
   • Fix: Add a delay or use sleep() in a loop:

     while (true) {
         $this->consume();
         sleep(5); // Poll every 5 seconds
     }
     

Extension Points

1. Custom Message Attributes Extend producers to add metadata:

   $this->producer->send('message', [
       'attribute1' => 'value1',
       'attribute2' => 'value2',
   ]);
   

2. Dynamic Queue Names Override getQueueName() in custom producers/consumers:

   class DynamicProducer implements ProducerInterface
   {
       public function getQueueName(): string
       {
           return 'queue_' . uniqid();
       }
   }
   

3. Retry Logic Implement exponential backoff in onError():

   private $retries = 0;
   private $maxRetries = 3;
   
   public function onError(string $message, \Exception $e): void
   {
       if ($this->retries++ < $this->maxRetries) {
           sleep(2 ** $this->retries); // Exponential backoff
           $this->consume(); // Re-consume
       }
   }
   

4. Cross-Account Access For cross-account SQS access, configure the queue policy to allow the external account’s IAM role.

Config Quirks

• Default Region: Falls back to us-east-1 if not specified in the service.
• Message Group ID: Required for FIFO queues. Set via producer arguments:

  arguments:
      $messageGroupId: 'group1'