Queue Manager Laravel Package

Getting Started

Minimal Steps

1. Installation

   composer require arko/queue-manager
   

   Register the service in your Laravel container (if using DI):

   $this->app->singleton(Arko\QueueManager\QueueManagerInterface::class, function ($app) {
       return new Arko\QueueManager\QueueManager();
   });
   

2. First Use Case: Delayed Task Execution

   // In a controller, command, or service
   $queueManager = app(Arko\QueueManager\QueueManagerInterface::class);
   $queueManager->add(function () {
       // Your logic here (e.g., send email, process data)
       Log::info('Task executed!');
   }, 'email-queue');
   
   // Later, trigger processing (e.g., via CLI or HTTP endpoint)
   $queueManager->process('email-queue');
   

3. Where to Look First

   • QueueManager class: Core logic for adding/processing queues.
   • QueueManagerInterface: Contract for dependency injection.
   • process() method: Key entry point for execution.

Implementation Patterns

Usage Patterns

1. Named Queues for Isolation Use distinct queue names (e.g., notifications, reports) to logically separate tasks. Example:

   $queueManager->add(fn() => $this->sendWelcomeEmail(), 'notifications');
   $queueManager->add(fn() => $this->generateReport(), 'reports');
   

2. Dependency Injection Bind the manager to your container (Laravel’s AppServiceProvider):

   public function register()
   {
       $this->app->bind(QueueManagerInterface::class, function () {
           return new QueueManager();
       });
   }
   

   Inject via constructor:

   public function __construct(private QueueManagerInterface $queueManager) {}
   

3. Contextual Processing Process queues in response to events or triggers:

   // In a controller after user signup
   $this->queueManager->add(fn() => $this->notifyAdmin(), 'admin-alerts');
   $this->queueManager->process('admin-alerts'); // Process immediately or defer
   

4. Closure-Based Tasks Leverage closures for scoped variables:

   $userId = 123;
   $queueManager->add(function () use ($userId) {
       $user = User::find($userId);
       // Task logic using $user
   }, 'user-tasks');
   

Workflows

1. CLI Processing Create an Artisan command to process queues:

   php artisan queue:process notifications
   

   (Extend QueueManager to support CLI flags.)

2. Webhook Triggers Expose an endpoint to process queues via HTTP:

   Route::post('/process-queue/{name}', function (string $name) {
       app(QueueManagerInterface::class)->process($name);
       return response()->json(['status' => 'processed']);
   });
   

3. Batch Processing Process multiple queues sequentially:

   $queues = ['emails', 'backups', 'logs'];
   foreach ($queues as $queue) {
       $this->queueManager->process($queue);
   }
   

Integration Tips

• Laravel Queues: Bridge with Laravel’s queue system by wrapping process() in a job:

  Queue::push(new ProcessQueueJob('queue-name'));
  

• Logging: Decorate QueueManager to log queue operations:

  $queueManager = new LoggingQueueManager(new QueueManager());
  

• Testing: Mock QueueManagerInterface in unit tests:

  $mock = Mockery::mock(QueueManagerInterface::class);
  $mock->shouldReceive('process')->once();
  

Gotchas and Tips

Pitfalls

1. Stateful Closures Avoid capturing mutable state (e.g., class properties) in closures unless intentional:

   // ❌ Risky: $this may change between adds/processes
   $queueManager->add(function () { $this->doSomething(); }, 'queue');
   
   // ✅ Safer: Pass dependencies explicitly
   $queueManager->add(fn() => $service->doSomething(), 'queue');
   

2. Queue Clearing Queues are cleared before processing, which may cause issues if:

   • A queue is processed mid-execution (e.g., nested calls).
   • You expect to reprocess the same queue without re-adding tasks. Workaround: Use a persistent storage (e.g., database) for critical queues.

3. No Retry Mechanism Failed tasks are not retried by default. Implement retry logic manually:

   $attempts = 0;
   $queueManager->add(function () use (&$attempts) {
       try {
           $this->riskyOperation();
       } catch (Exception $e) {
           if ($attempts++ < 3) {
               throw $e; // Requeue
           }
           Log::error("Failed after retries", ['error' => $e]);
       }
   }, 'retryable-queue');
   

4. Thread Safety The package is not thread-safe. Avoid concurrent access to the same QueueManager instance.

Debugging

1. Queue Inspection Add a debug method to inspect pending tasks:

   public function debug(string $queueName): array
   {
       return $this->queues[$queueName] ?? [];
   }
   

2. Error Handling Wrap process() calls to catch exceptions:

   try {
       $queueManager->process('critical-queue');
   } catch (Exception $e) {
       report($e);
       // Fallback logic
   }
   

3. Memory Leaks Large queues may consume memory. For long-running tasks:

   • Process queues in chunks.
   • Use Laravel’s queue system for heavy workloads.

Config Quirks

1. Singleton vs. Non-Singleton The package assumes singleton usage. If you need multiple instances:

   $instance1 = new QueueManager();
   $instance2 = new QueueManager(); // Separate queues
   

2. Queue Naming Use descriptive names (e.g., user-123-exports) to avoid collisions.

Extension Points

1. Custom Storage Extend QueueManager to use a database or Redis:

   class DatabaseQueueManager implements QueueManagerInterface {
       public function add(callable $callable, string $queueName) {
           DB::table('queues')->insert([...]);
       }
       public function process(string $queueName) {
           // Fetch from DB, execute, delete
       }
   }
   

2. Priority Queues Add a priority parameter to add() and sort queues accordingly:

   $queueManager->add($callable, 'high-priority', 1); // Lower number = higher priority
   

3. Event Dispatching Trigger events before/after processing:

   $queueManager->process('queue', fn() => event(new QueueProcessed('queue')));