Job Queue Bundle Laravel Package

## Getting Started

### Minimal Setup
1. **Installation**:
   ```bash
   composer require ajtis/job-queue-bundle

Enable the bundle in config/bundles.php:

return [
    // ...
    Ajtis\JobQueueBundle\AjtisJobQueueBundle::class => ['all' => true],
];

2. Configure the Bundle: Add the bundle configuration to config/packages/ajtis_job_queue.yaml:

   ajtis_job_queue:
       queue_name: 'default' # Default queue name
       workers_count: 4     # Number of concurrent workers
       log_file: '%kernel.logs_dir%/job_queue.log' # Worker log file
       command_prefix: 'app/console' # Prefix for console commands
   

3. First Use Case: Create a console command to test:

   php bin/console make:command TestJob
   

   Modify the generated command (src/Command/TestJobCommand.php) to accept arguments:

   protected function execute(InputInterface $input, OutputInterface $output): int
   {
       $output->writeln('Running test job with argument: ' . $input->getArgument('arg'));
       return Command::SUCCESS;
   }
   

   Enqueue the job via a controller or another command:

   use Ajtis\JobQueueBundle\JobQueue\JobQueueInterface;
   
   public function __construct(private JobQueueInterface $jobQueue) {}
   
   public function enqueueTestJob()
   {
       $this->jobQueue->addJob('test_job', ['arg' => 'test_value']);
   }
   

4. Run Workers: Start workers in a separate terminal:

   php bin/console ajtis:job-queue:worker
   

Implementation Patterns

Core Workflows

1. Job Enqueuing:

   • Use the JobQueueInterface to add jobs dynamically:

     $this->jobQueue->addJob('command_name', ['arg1' => 'value1', '--option' => 'value']);
     

   • For complex jobs, create a dedicated command and enqueue it with arguments/options.

2. Scheduling Jobs:

   • Use Symfony’s built-in scheduler (e.g., cron or symfony/scheduler) to trigger job enqueuing at specific times.
   • Example: Schedule a job to run daily via crontab:

     0 3 * * * php bin/console app:enqueue-daily-job
     

3. Worker Management:

   • Single Worker: Run one worker instance for development:

     php bin/console ajtis:job-queue:worker --env=dev
     

   • Multiple Workers: Scale horizontally by running multiple worker instances (e.g., in Docker/Kubernetes).
   • Graceful Shutdown: Workers handle SIGTERM/SIGINT gracefully. Use --timeout to limit job execution time:

     php bin/console ajtis:job-queue:worker --timeout=3600
     

4. Job Prioritization:

   • Use multiple queues (e.g., high, default, low) by configuring ajtis_job_queue per queue:

     ajtis_job_queue:
         queues:
             high:
                 workers_count: 2
             default:
                 workers_count: 4
     

   • Enqueue jobs to specific queues:

     $this->jobQueue->addJob('command_name', [], 'high');
     

5. Job Retries:

   • Configure retry logic in config/packages/ajtis_job_queue.yaml:

     ajtis_job_queue:
         retry:
             enabled: true
             max_attempts: 3
             delay: 60 # seconds between retries
     

   • Failed jobs are automatically retried with exponential backoff.

6. Monitoring:

   • Logs are written to log_file (configurable). Use monolog handlers to stream logs to external services (e.g., ELK, Datadog).
   • Add a custom command to list pending jobs:

     use Ajtis\JobQueueBundle\JobQueue\JobQueueInterface;
     
     public function listJobs(JobQueueInterface $jobQueue)
     {
         $jobs = $jobQueue->getPendingJobs();
         // Render output...
     }
     

Integration Tips

1. Dependency Injection:

   • Inject JobQueueInterface into services/controllers to enqueue jobs:

     use Ajtis\JobQueueBundle\JobQueue\JobQueueInterface;
     
     public function __construct(private JobQueueInterface $jobQueue) {}
     

2. Event-Driven Workflows:

   • Trigger jobs from Symfony events (e.g., KernelEvents::TERMINATE):

     use Symfony\Component\HttpKernel\Event\TerminateEvent;
     use Symfony\Component\HttpKernel\KernelEvents;
     
     public function onTerminate(TerminateEvent $event)
     {
         $this->jobQueue->addJob('cleanup_temp_files');
     }
     

3. Database-Backed Queues:

   • Use Doctrine to persist job metadata (e.g., status, retries) by extending the bundle’s Job entity:

     // src/Entity/Job.php
     use Ajtis\JobQueueBundle\Model\Job as BaseJob;
     
     class Job extends BaseJob
     {
         // Add custom fields (e.g., priority, tenant_id)
     }
     

   • Update the bundle’s configuration to use your custom entity.

4. Testing:

   • Mock JobQueueInterface in unit tests:

     $jobQueue = $this->createMock(JobQueueInterface::class);
     $jobQueue->expects($this->once())->method('addJob')->with('test_command', ['arg' => 'value']);
     

   • Use Ajtis\JobQueueBundle\Tests\JobQueueTestCase for integration tests.

5. Docker/Kubernetes:

   • Deploy workers as separate containers with resource limits:

     # docker-compose.yml
     services:
         job_worker:
             image: your-app
             command: php bin/console ajtis:job-queue:worker --env=prod
             deploy:
                 replicas: 3
     

Gotchas and Tips

Pitfalls

1. Worker Stuck in "Busy" State:

   • Cause: Long-running jobs or unhandled exceptions in commands.
   • Fix: Set --timeout in worker command or handle exceptions in commands:

     try {
         // Command logic
     } catch (\Exception $e) {
         throw new \RuntimeException('Job failed: ' . $e->getMessage());
     }
     

2. Jobs Disappearing Without Execution:

   • Cause: Workers not running or queue misconfiguration.
   • Debug: Check logs (log_file) and verify workers are connected to the correct queue name.

3. Argument/Option Parsing Issues:

   • Cause: Incorrect syntax when enqueuing jobs (e.g., mixing --option and arg).
   • Fix: Use the JobQueueInterface methods consistently:

     // Correct: Separate arguments and options
     $this->jobQueue->addJob('command_name', ['arg1', 'arg2'], [], ['--option' => 'value']);
     

4. Database Locking:

   • Cause: High concurrency with database-backed queues.
   • Fix: Configure doctrine connection pooling or use pessimistic_write locks in custom Job entities.

5. Worker Process Leaks:

   • Cause: Unclosed resources (e.g., files, database connections) in commands.
   • Fix: Ensure commands clean up resources and use finally blocks:

     $file = fopen('temp.txt', 'w');
     try {
         // Work with file
     } finally {
         fclose($file);
     }
     

Debugging Tips

1. Enable Verbose Logging:

   php bin/console ajtis:job-queue:worker --verbose
   

   Or configure monolog in config/packages/monolog.yaml:

   handlers:
       job_queue:
           type: stream
           path: '%kernel.logs_dir%/job_queue_verbose.log'
           level: debug
   

2. Inspect Queue State:

   • Use a custom command to dump pending jobs:

     public function listJobs(JobQueueInterface $jobQueue)
     {
         $jobs = $jobQueue->getPendingJobs();
         foreach ($jobs as $job) {
             var_dump($job->getCommand(), $job->getArguments(), $job->getOptions());
         }
     }
     

3. Simulate Failures:

   • Force a job to fail by throwing an exception in the command to test retries:

     throw new \RuntimeException('Simulated failure');
     

Configuration Quirks

1. Queue Name Overrides:
   • The queue_name in ajtis_job_queue.yaml is the default. Over