Queue System Bundle Laravel Package

Getting Started

Minimal Setup

1. Install Redis (if not already installed):

   wget http://download.redis.io/redis-stable.tar.gz
   tar xvzf redis-stable.tar.gz
   cd redis-stable
   make
   

   Start Redis server (src/redis-server) and ensure it’s running on localhost:6379.

2. Install the Bundle: Add to composer.json:

   "require": {
       "bernardosecades/queue-system-bundle": "dev-master"
   }
   

   Run:

   composer update
   

3. Register the Bundle: Add to AppKernel.php:

   new BernardoSecades\QueueSystemBundle\QueueSystemBundle(),
   

4. Configure Queues: Define queues in config.yml:

   queue_system:
       queues:
           images: "queue:mail"
           api: "queue:api"
   

5. First Use Case: Push a job to the api queue:

   $queue = $this->get('queue_system.queue.api');
   $queue->push('YourJobClass', ['param1' => 'value1']);
   

Implementation Patterns

Workflow: Job Creation and Processing

1. Define a Job Class: Implement a class with a run() method:

   class YourJobClass {
       public function run($data) {
           // Process data (e.g., send email, update DB)
           return true;
       }
   }
   

2. Push Jobs: Inject the queue service and push jobs:

   $queue = $this->get('queue_system.queue.api');
   $queue->push('YourJobClass', ['data' => 'payload']);
   

3. Process Jobs: Use a worker script (not provided; manually poll Redis):

   while true; do
       $queue = $this->get('queue_system.queue.api');
       $job = $queue->pop(); // Blocks until job is available
       $result = $job->run($job->getData());
   done
   

4. Dependency Injection: Access queues via Symfony’s DI container:

   $queue = $this->container->get('queue_system.queue.images');
   

Integration Tips

• Symfony Events: Trigger jobs from event listeners:

  $event->getSubject()->dispatch('queue_job', ['queue' => 'api', 'job' => 'YourJobClass']);
  

• Retry Logic: Implement retry in run() for transient failures.
• Monitoring: Use Redis CLI (redis-cli monitor) to debug job flow.

Gotchas and Tips

Pitfalls

1. No Built-in Worker: The bundle lacks a worker process. Use a loop (as shown above) or integrate with Symfony Messenger for production.

2. Hardcoded Redis Config: Host/port are hardcoded to localhost:6379 (per README). Override via DI or patch the bundle.

3. No Delayed Jobs: Jobs execute immediately after push(). For delays, use Redis’s ZADD or a wrapper.

4. Serializer Limitation: Only JSON serialization is supported. Custom serializers require bundle modification.

Debugging

• Check Redis:

  redis-cli LRANGE queue:api 0 -1  # List all jobs in the queue.
  

• Log Queue Actions: Extend the Queue class to log pushes/pops:

  class CustomQueue extends Queue {
      public function push($job, $data) {
          $this->logger->info("Pushed job: $job");
          parent::push($job, $data);
      }
  }
  

Extension Points

1. Custom Serializer: Override the serializer in QueueSystemBundle/Resources/config/services.yml:

   queue_system.serializer: '@your_custom.serializer'
   

2. Add Middleware: Extend the Queue class to add pre/post hooks:

   class HookedQueue extends Queue {
       public function push($job, $data) {
           $this->prePushHook($job, $data);
           parent::push($job, $data);
       }
   }
   

3. Queue Priorities: Use Redis lists with sorted sets (ZADD) for priority queues (not natively supported).