Job Queue Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require enqueue/job-queue
   

   Requires a transport (e.g., Redis, RabbitMQ, Doctrine DBAL). Install one first:

   composer require enqueue/redis
   

2. Configure Transport Add to config/queue.php (or create a custom config):

   'connections' => [
       'enqueue' => [
           'driver' => 'enqueue',
           'dsn' => 'redis://localhost',
       ],
   ],
   

3. First Job Define a job class:

   use Enqueue\Job\JobInterface;
   use Enqueue\Job\Job;
   
   class SendEmailJob implements JobInterface
   {
       public function __construct(private string $email) {}
   
       public function run(): void
       {
           // Logic to send email
       }
   
       public static function create(): JobInterface
       {
           return new self('user@example.com');
       }
   }
   

4. Dispatch a Job

   use Enqueue\Client\Producer;
   
   $producer = new Producer(new \Enqueue\Redis\RedisConnection('redis://localhost'));
   $producer->send(new SendEmailJob('user@example.com'));
   

5. Consume Jobs

   php artisan queue:work --queue=enqueue
   

   Or programmatically:

   use Enqueue\Client\Consumer;
   
   $consumer = new Consumer(new \Enqueue\Redis\RedisConnection('redis://localhost'));
   $consumer->consume(function (JobInterface $job) {
       $job->run();
       return true; // Acknowledge job completion
   });
   

Implementation Patterns

Common Workflows

1. Job Dependencies

Chain jobs with Job::createSubJob() or Job::createDependentJob():

$parentJob = new ParentJob();
$subJob = Job::createSubJob(new ChildJob(), $parentJob);
$producer->send($subJob);

2. Unique Jobs

Prevent duplicate execution with Job::createUniqueJob():

$uniqueJob = Job::createUniqueJob(
    new SendEmailJob('user@example.com'),
    'send_email_unique_key',
    3600 // TTL in seconds
);
$producer->send($uniqueJob);

3. Delayed Jobs

Schedule jobs for later:

$delayedJob = Job::createDelayedJob(new SendEmailJob('user@example.com'), new \DateTime('+1 hour'));
$producer->send($delayedJob);

4. Retry Logic

Implement retry logic in the job:

class RetryableJob implements JobInterface
{
    public function run(): void
    {
        try {
            // Risky operation
        } catch (Exception $e) {
            throw new Job\RetryableException($e->getMessage(), 3); // Retry 3 times
        }
    }
}

5. Laravel Integration

Use the enqueue/laravel bridge for seamless Laravel integration:

composer require enqueue/laravel

Publish config:

php artisan vendor:publish --provider="Enqueue\Laravel\LaravelServiceProvider"

Dispatch jobs like Laravel’s native queue:

dispatch(new SendEmailJob('user@example.com'));

Integration Tips

With Laravel

• Replace Illuminate\Queue\QueueManager with Enqueue\Laravel\QueueManager.
• Use Laravel’s dispatch() helper or Bus facade for job dispatching.
• Configure App\Providers\QueueServiceProvider to use Enqueue’s connection.

With Symfony

• Use Enqueue\Symfony\MessageBusFactory to integrate with Symfony Messenger.
• Configure transport in config/packages/enqueue.yaml.

Monitoring

• Use Enqueue\Client\Inspection\InspectionClient to inspect jobs:

  $inspector = new InspectionClient(new \Enqueue\Redis\RedisConnection('redis://localhost'));
  $jobs = $inspector->getJobs();
  

Performance

• Batch jobs for bulk processing:

  $producer->sendBatch([
      new SendEmailJob('user1@example.com'),
      new SendEmailJob('user2@example.com'),
  ]);
  

Gotchas and Tips

Pitfalls

1. Transport Compatibility

   • Ensure your transport (Redis, RabbitMQ, etc.) is running and accessible.
   • Misconfigured DSN (e.g., wrong host/port) will cause silent failures. Test with:

     $connection = new \Enqueue\Redis\RedisConnection('redis://localhost');
     if (!$connection->isConnected()) {
         throw new \RuntimeException('Transport not connected');
     }
     

2. Job Serialization

   • Jobs must be serializable. Avoid closures or non-serializable objects.
   • Use __serialize() and __unserialize() for custom serialization:

     public function __serialize(): array
     {
         return ['email' => $this->email];
     }
     
     public function __unserialize(array $data): void
     {
         $this->email = $data['email'];
     }
     

3. Acknowledgment

   • Always return true from the consumer callback to acknowledge job completion. Returning false or throwing an exception will retry the job (if configured).

4. Unique Job Keys

   • Unique jobs rely on a key. If the key is too generic (e.g., 'send_email'), collisions may occur. Use specific keys like 'send_email_user@example.com'.

5. Delayed Jobs

   • Delayed jobs require the transport to support delayed messages (e.g., Redis with delay plugin or RabbitMQ with x-delay header).

Debugging

1. Log Consumer Errors Wrap consumer logic in a try-catch:

   $consumer->consume(function (JobInterface $job) {
       try {
           $job->run();
           return true;
       } catch (Exception $e) {
           \Log::error('Job failed', ['job' => get_class($job), 'error' => $e->getMessage()]);
           return false; // Retry or dead-letter
       }
   });
   

2. Inspect Jobs Use the InspectionClient to debug stuck jobs:

   $inspector = new InspectionClient($connection);
   $jobs = $inspector->getJobs();
   foreach ($jobs as $job) {
       \Log::info('Pending job', ['job' => $job->getJobName()]);
   }
   

3. Transport-Specific Issues

   • Redis: Ensure the enqueue/redis package is installed and Redis is running.
   • RabbitMQ: Verify the queue exists and the user has permissions.
   • Doctrine DBAL: Check database connection and table structure (run migrations if using enqueue/doctrine).

Tips

1. Use Interfaces Extend JobInterface for all jobs to ensure consistency:

   class MyJob implements JobInterface { ... }
   

2. Leverage Middleware Add middleware for logging, metrics, or retries:

   $producer->setMiddleware([new \Enqueue\Middleware\LoggingMiddleware()]);
   

3. Dead-Letter Queues Configure dead-letter queues for failed jobs:

   $consumer->setMiddleware([
       new \Enqueue\Middleware\DeadLetterMiddleware('dead_letter_queue'),
   ]);
   

4. Testing Use Enqueue\Test\JobTestCase for unit tests:

   use Enqueue\Test\JobTestCase;
   
   class SendEmailJobTest extends JobTestCase
   {
       public function testJobExecution()
       {
           $job = new SendEmailJob('user@example.com');
           $this->assertTrue($job->run());
       }
   }
   

5. Custom Transports Extend Enqueue\Transport\TransportInterface for custom transports (e.g., Kafka, NATS).

6. Environment-Specific Config Use Laravel’s .env or Symfony’s parameters.yaml to switch transports between environments:

   QUEUE_CONNECTION=enqueue
   ENQUEUE_DSN=redis://localhost:6379
   

7. Job Priorities Use transport-specific features (e.g., RabbitMQ priorities) for job prioritization:

   $producer->send(new HighPriorityJob(), ['priority' => 1]);