Job Queue Bundle Laravel Package

## Getting Started

### Minimal Setup
1. **Installation**:
   ```bash
   composer require atexo/job-queue-bundle

Register the bundle in config/bundles.php:

return [
    // ...
    Atexo\JobQueueBundle\AtexoJobQueueBundle::class => ['all' => true],
];

2. Configure the Bundle: Update config/packages/atexo_job_queue.yaml (auto-generated after installation):

   atexo_job_queue:
       queue_name: 'default' # Default queue name
       storage:
           type: 'doctrine' # Options: 'doctrine', 'file', 'database'
           # Doctrine config (if using ORM)
           doctrine:
               entity: 'App\Entity\JobQueue' # Custom entity class
               connection: 'default'
           # File config (if using filesystem)
           file:
               directory: '%kernel.project_dir%/var/job_queue'
   

3. First Use Case: Annotate a Symfony command to run as a background job:

   use Atexo\JobQueueBundle\Annotation\Job;
   
   class MyBackgroundCommand extends Command
   {
       /**
        * @Job(queue="default", delay=60) // Runs in 60 seconds
        */
       protected function execute(InputInterface $input, OutputInterface $output): int
       {
           // Your command logic
           return Command::SUCCESS;
       }
   }
   

4. Run the Worker: Start the job queue worker in a separate terminal:

   php bin/console atexo:job-queue:worker
   

Implementation Patterns

Common Workflows

1. Scheduling Jobs:

   • Use the @Job annotation to define queue and delay:

     /**
      * @Job(queue="high_priority", delay=3600) // Runs in 1 hour
      */
     

   • Dynamically schedule jobs via the JobQueueClient service:

     $client = $container->get('atexo.job_queue.client');
     $client->schedule('app:my_command', ['arg1' => 'value'], 'default', 60);
     

2. Handling Job Results:

   • Override the JobQueueListener to process job results:

     use Atexo\JobQueueBundle\Event\JobEvent;
     
     public function onJobCompleted(JobEvent $event)
     {
         if ($event->getExitCode() !== 0) {
             // Handle failure
         }
     }
     

   • Register the listener in services.yaml:

     services:
         App\EventListener\JobListener:
             tags:
                 - { name: 'kernel.event_listener', event: 'atexo.job_queue.job.completed', method: 'onJobCompleted' }
     

3. Retry Logic:

   • Configure retries in config/packages/atexo_job_queue.yaml:

     atexo_job_queue:
         retry:
             enabled: true
             max_attempts: 3
             delay: 300 # 5 minutes between retries
     

4. Bulk Processing:

   • Use the batch option to group jobs:

     /**
      * @Job(queue="batch", batch=true, batch_size=10)
      */
     

   • Process batches in the worker by implementing BatchAwareInterface.

5. Dependency Injection:

   • Inject services into commands:

     public function __construct(private MyService $myService) {}
     
     /**
      * @Job(queue="default")
      */
     protected function execute(InputInterface $input, OutputInterface $output): int
     {
         $this->myService->doSomething();
         return Command::SUCCESS;
     }
     

Integration Tips

1. With Doctrine:

   • Ensure your JobQueue entity extends Atexo\JobQueueBundle\Entity\JobQueue:

     use Atexo\JobQueueBundle\Entity\JobQueue as BaseJobQueue;
     
     class JobQueue extends BaseJobQueue {}
     

   • Update config/packages/doctrine.yaml to include the entity:

     orm:
         mappings:
             app:
                 type: annotation
                 dir: '%kernel.project_dir%/src/Entity'
                 prefix: 'App\Entity'
                 alias: App
     

2. With Supervisor:

   • Configure a Supervisor process to keep the worker running:

     [program:jobqueue-worker]
     command=php /path/to/bin/console atexo:job-queue:worker
     autostart=true
     autorestart=true
     user=www-data
     numprocs=1
     redirect_stderr=true
     stderr_logfile=/var/log/jobqueue-worker.err.log
     

3. With Symfony Messenger:

   • Bridge jobs to Messenger for advanced routing:

     # config/packages/messenger.yaml
     framework:
         messenger:
             transports:
                 async: '%env(MESSENGER_TRANSPORT_DSN)%'
             routing:
                 'Atexo\JobQueueBundle\Message\JobMessage': async
     

4. Testing:

   • Mock the JobQueueClient in tests:

     $client = $this->createMock(JobQueueClient::class);
     $client->expects($this->once())
            ->method('schedule')
            ->with('app:command', ['arg' => 'value'], 'queue', 0);
     $this->container->set('atexo.job_queue.client', $client);
     

Gotchas and Tips

Pitfalls

1. Worker Stuck on Jobs:

   • Cause: Unhandled exceptions in commands or infinite loops.
   • Fix: Implement a JobListener to log failures and ensure commands exit properly:

     public function onJobFailed(JobEvent $event)
     {
         $this->logger->error('Job failed', [
             'command' => $event->getCommand(),
             'exit_code' => $event->getExitCode(),
             'output' => $event->getOutput(),
         ]);
     }
     

2. Queue Not Processing:

   • Cause: Worker not running or misconfigured storage.
   • Debug:
     • Check logs: var/log/jobqueue-worker.log.
     • Verify storage directory permissions (for file storage).
     • Ensure Doctrine connection is healthy (for doctrine storage).

3. Annotation Not Recognized:

   • Cause: Missing jms_serializer or incorrect annotation import.
   • Fix: Install jms/serializer-bundle and ensure annotations are imported:

     use Atexo\JobQueueBundle\Annotation\Job;
     

4. Delayed Jobs Not Respecting Delay:

   • Cause: Worker polling too frequently or system clock skew.
   • Fix: Adjust polling_interval in config:

     atexo_job_queue:
         polling_interval: 10 # seconds
     

5. Memory Leaks:

   • Cause: Long-running commands or unclosed resources.
   • Fix: Use setTimeout in the annotation or config:

     atexo_job_queue:
         timeout: 300 # 5 minutes
     

Debugging Tips

1. Enable Verbose Logging:

   php bin/console atexo:job-queue:worker --verbose
   

   Or in config/packages/monolog.yaml:

   handlers:
       job_queue:
           type: stream
           path: '%kernel.logs_dir%/job_queue.log'
           level: debug
   

2. Inspect Queue State:

   • For Doctrine storage:

     php bin/console doctrine:query:sql "SELECT * FROM job_queue"
     

   • For file storage:

     ls -la %kernel.project_dir%/var/job_queue/
     

3. Simulate Failures:

   • Force a job to fail by throwing an exception in the command:

     throw new \RuntimeException('Test failure');
     

Extension Points

1. Custom Storage:

   • Implement Atexo\JobQueueBundle\Storage\StorageInterface:

     class CustomStorage implements StorageInterface
     {
         public function enqueue(Job $job): void { /* ... */ }
         public function dequeue(): ?Job { /* ... */ }
         public function delete(Job $job): void { /* ... */ }
     }
     

   • Register in config/packages/atexo_job_queue.yaml:

     atexo_job_queue:
         storage:
             type: 'custom'
             service: 'app.custom_storage'
     

2. Custom Job Entity:

   • Extend Atexo\JobQueueBundle\Entity\JobQueue and add fields:

     /**
      * @ORM\Entity
      */
     class CustomJobQueue extends JobQueue
     {
         /**
          * @ORM\Column(type="string", nullable=true)
          */
         private $customField;
     }