Job Queue Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require effiana/job-queue-bundle
   

   Enable the bundle in config/bundles.php:

   return [
       // ...
       Effiana\JobQueueBundle\EffianaJobQueueBundle::class => ['all' => true],
   ];
   

2. Configure the Bundle: Update config/packages/effiana_job_queue.yaml with your queue driver (e.g., doctrine, database, or redis):

   effiana_job_queue:
       driver: doctrine
       connection: default
       table: job_queue
   

3. First Use Case: Convert a Symfony command to a background job:

   // src/Command/MyCommand.php
   namespace App\Command;
   
   use Symfony\Component\Console\Command\Command;
   use Symfony\Component\Console\Input\InputInterface;
   use Symfony\Component\Console\Output\OutputInterface;
   use Effiana\JobQueueBundle\JobQueue\JobQueueInterface;
   
   class MyCommand extends Command
   {
       protected static $defaultName = 'app:my-command';
   
       private $jobQueue;
   
       public function __construct(JobQueueInterface $jobQueue)
       {
           parent::__construct();
           $this->jobQueue = $jobQueue;
       }
   
       protected function execute(InputInterface $input, OutputInterface $output): int
       {
           $this->jobQueue->addJob(
               'app:my-command',
               ['--option' => 'value'], // Command arguments
               'now' // Schedule time (e.g., '+1 hour' for delayed)
           );
           return Command::SUCCESS;
       }
   }
   

4. Run the Queue Worker:

   php bin/console effiana:job-queue:consume
   

Implementation Patterns

Workflows

1. Scheduling Jobs:

   • Use cron-like syntax for delayed jobs:

     $this->jobQueue->addJob('app:command', [], '+10 minutes');
     

   • Schedule recurring jobs via Symfony's CronExpression:

     $this->jobQueue->addJob('app:daily-report', [], '0 0 * * *');
     

2. Dependency Injection: Inject JobQueueInterface into services to queue commands dynamically:

   public function __construct(JobQueueInterface $jobQueue) {
       $this->jobQueue = $jobQueue;
   }
   

3. Job Prioritization: Use the priority option (lower = higher priority):

   $this->jobQueue->addJob('app:high-priority', [], 'now', 1);
   

4. Handling Output/Errors: Redirect command output to a file or log:

   # config/packages/effiana_job_queue.yaml
   effiana_job_queue:
       log_output: true
       log_file: '%kernel.logs_dir%/job_queue.log'
   

Integration Tips

• Laravel Compatibility: If using Laravel, alias the bundle in config/app.php:

  'aliases' => [
      'JobQueue' => Effiana\JobQueueBundle\JobQueue\JobQueue::class,
  ],
  

  Then inject JobQueue instead of JobQueueInterface.

• Queue Workers in Production: Run workers in a separate process (e.g., Supervisor):

  [program:jobqueue]
  command=php /path/to/bin/console effiana:job-queue:consume
  autostart=true
  autorestart=true
  user=www-data
  numprocs=4
  

• Custom Job Classes: Extend Effiana\JobQueueBundle\JobQueue\Job for custom logic:

  namespace App\JobQueue;
  
  use Effiana\JobQueueBundle\JobQueue\Job;
  
  class CustomJob extends Job
  {
      public function __construct(string $command, array $arguments = [], string $schedule = 'now', int $priority = 0)
      {
          parent::__construct($command, $arguments, $schedule, $priority);
          $this->setCustomProperty('key', 'value');
      }
  }
  

Gotchas and Tips

Pitfalls

1. Database Locking:

   • The doctrine driver uses database locks for job processing. Ensure your database supports SELECT ... FOR UPDATE.
   • Fix: Use redis or database driver for high-concurrency environments.

2. Command Arguments:

   • Arguments must be serializable. Avoid passing objects or closures directly.
   • Workaround: Use IDs or serialized strings.

3. Worker Stuck on Jobs:

   • If a worker crashes mid-job, the job may remain locked.
   • Solution: Run php bin/console effiana:job-queue:release to free stuck jobs.

4. Timezone Issues:

   • Schedule times use the server's timezone. Explicitly set timezone in config/packages/framework.yaml:

     framework:
         timezone: UTC
     

Debugging

• Check Job Status:

  php bin/console effiana:job-queue:list
  

• View Worker Logs:

  tail -f var/log/job_queue.log
  

• Clear Failed Jobs:

  php bin/console effiana:job-queue:purge
  

Config Quirks

• Driver-Specific Settings:

  • Doctrine: Requires job_queue table. Run migrations:

    php bin/console doctrine:migrations:diff
    php bin/console doctrine:migrations:migrate
    

  • Redis: Ensure predis/predis is installed:

    composer require predis/predis
    

• Environment Variables: Override settings via .env:

  JOB_QUEUE_DRIVER=redis
  JOB_QUEUE_REDIS_URL=redis://localhost:6379
  

Extension Points

1. Custom Job Storage: Implement Effiana\JobQueueBundle\JobQueue\JobRepositoryInterface for custom storage (e.g., Elasticsearch).

2. Event Listeners: Subscribe to job events (e.g., JobStarted, JobFailed):

   // src/EventListener/JobListener.php
   namespace App\EventListener;
   
   use Effiana\JobQueueBundle\Event\JobEvent;
   use Symfony\Component\EventDispatcher\EventSubscriberInterface;
   
   class JobListener implements EventSubscriberInterface
   {
       public static function getSubscribedEvents()
       {
           return [
               'job.started' => 'onJobStarted',
               'job.failed' => 'onJobFailed',
           ];
       }
   
       public function onJobStarted(JobEvent $event) {
           // Log job start
       }
   
       public function onJobFailed(JobEvent $event) {
           // Handle failures (e.g., send Slack alert)
       }
   }
   

3. Retry Logic: Configure retries in config/packages/effiana_job_queue.yaml:

   effiana_job_queue:
       max_retries: 3
       retry_delay: 300 # 5 minutes