Batch Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require akeneo/batch-bundle
   

   Ensure your AppKernel.php includes the bundle:

   new Akeneo\BatchBundle\AkeneoBatchBundle(),
   

2. First Job Definition: Define a job in YAML (config/batch/jobs.yml):

   akeneo_batch:
       jobs:
           my_first_job:
               type: chain
               steps:
                   - name: step1
                     type: doctrine
                     reader:
                         query: SELECT u FROM AppBundle\Entity\User u
                     processor:
                         class: AppBundle\Processor\UserProcessor
                     writer:
                         class: AppBundle\Writer\UserWriter
   

3. Run the Job:

   php app/console akeneo:batch:run my_first_job
   

4. Check Status:

   php app/console akeneo:batch:list-jobs
   

First Use Case: Importing CSV Data

1. Define a CSV Reader:

   reader:
       class: AppBundle\Reader\CsvReader
       arguments: ["%kernel.root_dir%/../data/import.csv"]
   

   Implement AppBundle\Reader\CsvReader extending Akeneo\Batch\Reader\AbstractReader.

2. Process and Write:

   processor:
       class: AppBundle\Processor\ProductProcessor
   writer:
       class: AppBundle\Writer\DoctrineWriter
       arguments: ["@doctrine.orm.entity_manager"]
   

3. Run and Monitor: Use the CLI commands to execute and track progress via JobExecution logs.

Implementation Patterns

Job Design Patterns

1. Chain Jobs:

   type: chain
   steps:
       - name: validate
         type: doctrine
         # ...
       - name: process
         type: chain
         steps:
             - name: transform
               # ...
   

2. Conditional Steps: Use Akeneo\Batch\Step\StepExecutionListenerInterface to skip steps based on conditions:

   public function onStepStart(StepExecution $stepExecution) {
       if (!$stepExecution->getJobExecution()->isRunning()) {
           $stepExecution->markAsSkipped();
       }
   }
   

Integration with Symfony Services

1. Dependency Injection: Inject services into readers/processors/writers:

   processor:
       class: AppBundle\Processor\OrderProcessor
       arguments: ["@service_container.get('app.mailer')"]
   

2. Event Dispatching: Listen to batch events (e.g., JobStartEvent, StepFailEvent):

   services:
       app.batch_listener:
           class: AppBundle\EventListener\BatchListener
           tags:
               - { name: kernel.event_listener, event: akeneo_batch.job_start, method: onJobStart }
   

Common Workflows

1. Bulk Data Operations:

   • Use doctrine step type for database operations.
   • Example: Bulk update product prices:

     reader:
         query: SELECT p FROM AppBundle\Entity\Product p WHERE p.price < 100
     processor:
         class: AppBundle\Processor\PriceUpdateProcessor
     writer:
         class: AppBundle\Writer\DoctrineWriter
     

2. Scheduled Jobs: Use Symfony’s CronBundle or Laravel Task Scheduling to trigger jobs:

   # config.yml
   akeneo_batch:
       jobs:
           nightly_export:
               schedule: "0 3 * * *"  # Runs daily at 3 AM
   

3. Retry Failed Jobs: Implement Akeneo\Batch\Job\JobExecutionListenerInterface to handle retries:

   public function onJobFail(JobExecution $jobExecution) {
       $jobExecution->markAsRestartable();
   }
   

Gotchas and Tips

Pitfalls

1. Deprecated Methods: Avoid getEntityManager() in processors/readers/writers. Use dependency injection instead:

   // ❌ Avoid
   $em = $this->getEntityManager();
   
   // ✅ Prefer
   public function __construct(EntityManager $em) { $this->em = $em; }
   

2. Performance Issues:

   • Large datasets may cause memory leaks. Use chunking in readers/writers:

     reader:
         chunk_size: 100  # Process 100 items at a time
     

   • Monitor JobExecution logs for slow steps.

3. Job Isolation:

   • Jobs run in the same process by default. For long-running tasks, consider:
     • Running jobs in separate processes (e.g., Symfony’s Process component).
     • Using a message queue (e.g., Symfony Messenger) to decouple jobs.

Debugging Tips

1. Verbose Output: Run jobs with -v for detailed logs:

   php app/console akeneo:batch:run my_job -v
   

2. Step-Level Debugging: Add debug listeners to inspect StepExecution:

   public function onStepStart(StepExecution $stepExecution) {
       \Log::debug('Step started:', [
           'id' => $stepExecution->getId(),
           'status' => $stepExecution->getStatus(),
       ]);
   }
   

3. Database Locks:

   • Use JobExecution locks to prevent concurrent runs:

     akeneo_batch:
         jobs:
             my_job:
                 lock: true  # Ensures only one instance runs at a time
     

Extension Points

1. Custom Step Types: Extend Akeneo\Batch\Step\AbstractStep to create reusable step types (e.g., api for HTTP calls).

2. Custom Writers: Implement Akeneo\Batch\Writer\WriterInterface for non-Doctrine storage (e.g., Elasticsearch):

   class ElasticsearchWriter implements WriterInterface {
       public function write(array $items) {
           $this->client->bulk($items);
       }
   }
   

3. Job Metadata: Store custom metadata in JobExecution:

   $jobExecution->addJobParameter('source_file', 'data/import.csv');
   

Configuration Quirks

1. YAML vs. XML: The bundle primarily supports YAML for job definitions. For XML, use Symfony’s Extension system to parse custom formats.

2. Job Naming: Use hyphenated, lowercase names (e.g., export-products) for CLI commands and logs.

3. Doctrine Integration: Ensure your EntityManager is configured for batch operations:

   doctrine:
       orm:
           dql:
               string_functions:
                   CONCAT: AppBundle\DQL\Concat