Job Queue Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require aureja/job-queue-bundle "dev-master"
   

   Add the bundle to AppKernel.php under registerBundles().

2. Entity Setup: Extend Aureja\JobQueue\Model\JobReport and Aureja\JobQueue\Model\JobConfiguration in your src/ directory (e.g., src/Acme/YourBundle/Entity/). Example:

   // src/Acme/YourBundle/Entity/JobReport.php
   namespace Acme\YourBundle\Entity;
   use Aureja\JobQueue\Model\JobReport as BaseJobReport;
   use Doctrine\ORM\Mapping as ORM;
   
   /**
    * @ORM\Entity
    * @ORM\Table(name="aureja_job_report")
    */
   class JobReport extends BaseJobReport { ... }
   

3. Configuration: Update config.yml with your entity classes and queues:

   aureja_job_queue:
       db_driver: orm
       class:
           model:
               job_configuration: Acme\YourBundle\Entity\JobConfiguration
               job_report: Acme\YourBundle\Entity\JobReport
       queues:
           - default
   

4. Database Schema: Run php app/console doctrine:schema:update --force to create tables.

5. Routing: Import routes in routing.yml:

   aureja_job_queue:
       resource: "@AurejaJobQueueBundle/Resources/config/routing.xml"
   

6. First Job: Enqueue a job via the CLI or a controller:

   php app/console aureja:job-queue:enqueue Acme\YourBundle\Command\YourJobCommand
   

Implementation Patterns

Core Workflows

1. Job Enqueuing: Use the enqueue command to dispatch jobs to queues:

   php app/console aureja:job-queue:enqueue Acme\YourBundle\Command\ProcessUserCommand --queue=default --data='{"user_id": 1}'
   

   • Dynamic Data: Pass serialized data (JSON/array) via --data.
   • Queue Selection: Specify --queue (e.g., high_priority).

2. Job Processing: Register a cronjob to run the worker every minute:

   * * * * * php app/console aureja:job-queue:run
   

   • Manual Trigger: Run aureja:job-queue:run ad-hoc for testing.

3. Job Monitoring: Use the admin interface (auto-routed) to:

   • View pending/failed jobs.
   • Retry or delete jobs via the UI.

4. Custom Job Logic: Extend Aureja\JobQueue\Model\Job or implement Aureja\JobQueue\Model\JobInterface:

   namespace Acme\YourBundle\Job;
   use Aureja\JobQueue\Model\Job as BaseJob;
   
   class ProcessUserJob extends BaseJob {
       public function execute() {
           // Custom logic (e.g., fetch user from DB, process data).
           return $this->success(); // or $this->fail('Error message');
       }
   }
   

5. Configuration Management:

   • Queue-Specific Settings: Define retry limits, timeouts, or priority in JobConfiguration.
   • Dynamic Queues: Add/remove queues via config.yml without code changes.

Integration Tips

1. Symfony Commands: Dispatch jobs from commands:

   use Aureja\JobQueueBundle\Command\EnqueueCommand;
   $enqueue = $this->getContainer()->get('aureja.job_queue.enqueue');
   $enqueue->enqueue('Acme\YourBundle\Job\ProcessUserJob', ['user_id' => 1], 'default');
   

2. Events: Listen for job lifecycle events (e.g., job.enqueued, job.failed) via Symfony’s event dispatcher.

3. Testing:

   • Mock the JobQueue service in PHPUnit:

     $jobQueue = $this->getMockBuilder('Aureja\JobQueue\JobQueue')
         ->disableOriginalConstructor()
         ->getMock();
     $this->container->set('aureja.job_queue', $jobQueue);
     

   • Use aureja:job-queue:clear to reset test data.

4. Performance:

   • Batch jobs with JobConfiguration::setBatchSize().
   • Use separate queues for high-priority tasks (e.g., critical).

Gotchas and Tips

Pitfalls

1. Cronjob Misconfiguration:

   • Ensure the cronjob runs as the correct user (e.g., www-data for Symfony).
   • Test manually first: php app/console aureja:job-queue:run --env=dev.

2. Entity Mapping Issues:

   • If extending JobReport/JobConfiguration, ensure all @ORM\ annotations match the base class.
   • Fix: Compare with Aureja\JobQueue\Model\JobReport annotations.

3. Queue Locking:

   • Jobs may hang if the worker crashes. Use JobConfiguration::setTimeout() to auto-release locks.

4. Data Serialization:

   • Only JSON-serializable data can be passed to jobs. Avoid passing:
     • Objects (use IDs or DTOs).
     • Resources (e.g., file handles).

5. Database Driver:

   • The bundle defaults to orm. For custom drivers (e.g., MongoDB), implement Aureja\JobQueue\Driver\DriverInterface.

Debugging Tips

1. Log Output: Enable debug mode (APP_DEBUG=true) to see job execution logs in var/log/dev.log.

2. Admin Panel:

   • Use the auto-generated UI to inspect job statuses and errors.
   • Tip: Filter by queue or status (e.g., failed).

3. Command-Line Flags:

   • Dry-run jobs with --dry-run:

     php app/console aureja:job-queue:run --dry-run
     

   • Verbose output:

     php app/console aureja:job-queue:run -v
     

4. Common Errors:

   • "Job not found": Verify the job class implements JobInterface and is autoloaded.
   • Doctrine errors: Run php app/console doctrine:schema:validate to check entity mappings.

Extension Points

1. Custom Drivers: Implement Aureja\JobQueue\Driver\DriverInterface for non-Doctrine storage (e.g., Redis):

   class RedisDriver implements DriverInterface {
       public function save(JobReport $report) { ... }
       public function findPending() { ... }
   }
   

   Register in services.yml:

   aureja.job_queue.driver: '@your.redis_driver'
   

2. Job Middleware: Add pre/post-processing logic by extending Aureja\JobQueue\JobQueue and overriding executeJob().

3. Event Listeners: Subscribe to job events (e.g., job.failed) to trigger alerts or retries:

   // src/Acme/YourBundle/EventListener/JobListener.php
   namespace Acme\YourBundle\EventListener;
   use Aureja\JobQueue\Event\JobEvent;
   
   class JobListener {
       public function onJobFailed(JobEvent $event) {
           // Send Slack notification, etc.
       }
   }
   

   Register in services.yml:

   services:
       acme.job_listener:
           class: Acme\YourBundle\EventListener\JobListener
           tags:
               - { name: kernel.event_listener, event: job.failed, method: onJobFailed }
   

4. Custom Job Statuses: Extend the status field in JobReport (e.g., processing, retrying) and update the admin template.