Sqs Command Queue Bundle Laravel Package

Getting Started

Minimal Steps

1. Installation Run:

   composer require bbit/sqs-command-queue-bundle:dev-master
   

   (Note: Use dev-master as the package is outdated; pin to a specific commit if needed.)

2. Enable Bundle Add to app/AppKernel.php:

   new BBIT\SqsCommandQueueBundle\BBITSqsCommandQueueBundle(),
   

3. Configure Add to app/config/config.yml:

   bbit_sqs_command_queue:
       aws_sqs_key: "%aws_sqs_key%"
       aws_sqs_secret: "%aws_sqs_secret%"
       aws_sqs_region: "eu-central-1"
       aws_sqs_queue: "https://sqs.eu-central-1.amazonaws.com/xx11xx11/xx-xx"
   

   Store credentials in .env or parameters.yml (never commit secrets).

4. First Use Case Dispatch a command via a service:

   $this->get('sqs_queue')->addCommand('app/console cache:clear');
   

   Or directly via container:

   php app/console sqs:send-command "app/console cache:clear"
   

Implementation Patterns

Core Workflows

1. Dispatching Commands

   • From Controllers/Commands:

     $this->get('sqs_queue')->addCommand('app/console my:command --option=value');
     

   • From Event Listeners:

     public function onKernelRequest(GetResponseEvent $event) {
         $this->container->get('sqs_queue')->addCommand('app/console my:async-task');
     }
     

   • Bulk Dispatch: Use a loop or array_map to queue multiple commands (e.g., for batch processing).

2. Worker Management

   • Run Worker:

     php app/console bbit:sqs:worker
     

     (Run in a separate process/terminal; use screen/tmux for persistence.)
   • Scaling: Deploy multiple workers (e.g., via PM2, Supervisor, or Kubernetes) to parallelize tasks. Example Supervisor config:

     [program:sqs-worker]
     command=php /path/to/app/console bbit:sqs:worker
     autostart=true
     autorestart=true
     user=www-data
     

3. Command Integration

   • Custom Commands: Extend Symfony’s Command class and queue them like built-in commands. Example:

     // src/AppBundle/Command/MyAsyncCommand.php
     namespace AppBundle\Command;
     use Symfony\Component\Console\Command\Command;
     class MyAsyncCommand extends Command { ... }
     

     Queue with:

     $this->get('sqs_queue')->addCommand('app:my-async-command');
     

4. Error Handling

   • Retry Logic: Implement BBIT\SqsCommandQueueBundle\Command\CommandInterface to add retry logic for failed commands.
   • Dead Letter Queue (DLQ): Configure SQS DLQ in AWS for failed messages (requires manual setup in SQS console).

5. Monitoring

   • CloudWatch: Enable SQS metrics in AWS CloudWatch to track queue depth/latency.
   • Logging: Extend the worker to log command execution:

     // Override BBIT\SqsCommandQueueBundle\Worker\Worker
     public function executeCommand($command) {
         $this->logger->info("Executing: $command");
         parent::executeCommand($command);
     }
     

Gotchas and Tips

Pitfalls

1. Outdated Package

   • Issue: Last release in 2015; lacks PHP 7.4+/8.x support.
   • Workaround:
     • Fork the repo and update dependencies (e.g., aws/aws-sdk-php to v3).
     • Use a wrapper service (e.g., league/flysystem-aws-s3-v3) for SQS if needed.
   • Alternative: Consider modern alternatives like:
     • symfony/messenger + symfony/amqp-messenger for RabbitMQ.
     • php-amqplib for RabbitMQ/SQS via STOMP.

2. AWS Credentials

   • Issue: Hardcoding secrets in config.yml is unsafe.
   • Fix: Use .env files or AWS IAM roles (for EC2 instances). Example .env:

     AWS_SQS_KEY=AKIAXXXXXXXXXXXXXXXX
     AWS_SQS_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
     

3. Worker Stuck on Failures

   • Issue: Worker may hang if SQS visibility timeout expires or commands fail silently.
   • Debugging:
     • Check SQS metrics for ApproximateNumberOfMessagesNotVisible.
     • Add logging to the worker:

       // Override BBIT\SqsCommandQueueBundle\Worker\Worker
       public function executeCommand($command) {
           try {
               parent::executeCommand($command);
           } catch (\Exception $e) {
               $this->logger->error("Failed to execute $command: " . $e->getMessage());
               throw $e; // Re-throw to trigger SQS retry
           }
       }
       

4. Visibility Timeout

   • Issue: Default SQS visibility timeout (30s) may cause duplicate processing if workers are slow.
   • Fix: Increase timeout in AWS SQS console (max 12 hours) or optimize command execution.

5. Command Timeouts

   • Issue: Long-running commands may time out or block workers.
   • Solution: Break commands into smaller steps or use Symfony’s Process component for subprocesses.

Tips

1. Environment-Specific Queues Use different SQS queues per environment (e.g., dev-queue, prod-queue):

   # config_dev.yml
   bbit_sqs_command_queue:
       aws_sqs_queue: "https://sqs.eu-central-1.amazonaws.com/xx11xx11/dev-queue"
   

2. Priority Queues

   • Workaround: Use SQS FIFO queues (if available) or prefix commands with priority:

     $this->get('sqs_queue')->addCommand('high:app:cache:clear');
     

     Then filter in the worker:

     if (strpos($command, 'high:') === 0) {
         $this->executeHighPriority($command);
     }
     

3. Testing

   • Local SQS: Use LocalStack for testing:

     docker run -it -p 4566:4566 localstack/localstack
     

     Update config to point to LocalStack’s SQS endpoint.

4. Performance Tuning

   • Batch Processing: Process multiple messages in one worker loop (if commands are independent). Example override:

     public function run() {
         while ($messages = $this->sqs->receiveMessages()) {
             foreach ($messages as $message) {
                 $this->executeCommand($message->getBody());
                 $this->sqs->deleteMessage($message);
             }
         }
     }
     

5. Security

   • IAM Policies: Restrict SQS access to only necessary actions (e.g., sqs:SendMessage, sqs:ReceiveMessage).
   • Encryption: Enable SQS encryption at rest in AWS console.

6. Extending the Bundle

   • Custom Worker: Subclass BBIT\SqsCommandQueueBundle\Worker\Worker to add features like:
     • Command blacklisting.
     • Dynamic scaling based on queue depth.
   • Event Listeners: Hook into sqs.command.send and sqs.command.execute events (if the bundle supports them; may require patching).