Batch Laravel Package

Getting Started

Minimal Steps to Begin

1. Installation

   composer require akeneo/batch
   

   Add the service provider to config/app.php:

   'providers' => [
       // ...
       Akeneo\Batch\JobRunner\JobRunnerServiceProvider::class,
   ],
   

2. Define a Job Create a job class extending Akeneo\Batch\Job\JobInterface:

   namespace App\Jobs;
   
   use Akeneo\Batch\Job\JobInterface;
   use Akeneo\Batch\Step\StepExecution;
   use Akeneo\Batch\Step\StepInterface;
   
   class MyJob implements JobInterface
   {
       public function getName(): string
       {
           return 'my_job';
       }
   
       public function run(StepExecution $stepExecution): void
       {
           // Job logic here
       }
   }
   

3. Register the Job Define the job in config/batch.php:

   'jobs' => [
       'my_job' => [
           'class' => \App\Jobs\MyJob::class,
           'steps' => [
               // Define steps here (see Implementation Patterns)
           ],
       ],
   ],
   

4. Run the Job Use the JobRunner facade:

   use Akeneo\Batch\JobRunner\JobRunner;
   
   $jobRunner = app(JobRunner::class);
   $jobRunner->run('my_job');
   

5. Check Documentation Focus on:

   • JobInterface
   • StepInterface
   • JobRunner

Implementation Patterns

Core Workflow: Job → Steps → Chunks

1. Job Definition Jobs are the top-level unit of work. Define them in config/batch.php with:

   • Class: The job class implementing JobInterface.
   • Steps: An array of step configurations (see below).
   • Retry Limit: Defaults to 3 (configurable per job).
   • Chunk Size: Defaults to 100 (configurable per step).

   'jobs' => [
       'import_products' => [
           'class' => \App\Jobs\ImportProductsJob::class,
           'steps' => [
               'read' => [
                   'class' => \App\Steps\ReadFromCsvStep::class,
                   'reader' => 'csv_reader',
               ],
               'process' => [
                   'class' => \App\Steps\ProcessProductStep::class,
               ],
               'write' => [
                   'class' => \App\Steps\WriteToDatabaseStep::class,
                   'writer' => 'database_writer',
               ],
           ],
       ],
   ],
   

2. Step Patterns Steps are the building blocks of a job. Common patterns:

   • Reader Steps: Fetch data (e.g., from CSV, API, DB).

     class ReadFromCsvStep implements StepInterface
     {
         public function run(StepExecution $stepExecution): void
         {
             $reader = $this->getReader($stepExecution);
             while ($reader->hasNext()) {
                 $data = $reader->next();
                 $stepExecution->addReadData($data);
             }
         }
     }
     

   • Processor Steps: Transform data.

     class ProcessProductStep implements StepInterface
     {
         public function run(StepExecution $stepExecution): void
         {
             $chunk = $stepExecution->getReadData();
             foreach ($chunk as $item) {
                 $processed = $this->transform($item);
                 $stepExecution->addProcessedData($processed);
             }
         }
     }
     

   • Writer Steps: Persist data.

     class WriteToDatabaseStep implements StepInterface
     {
         public function run(StepExecution $stepExecution): void
         {
             $chunk = $stepExecution->getProcessedData();
             foreach ($chunk as $item) {
                 $this->saveToDatabase($item);
             }
         }
     }
     

3. Chunking Use StepExecution::getReadData() and StepExecution::getProcessedData() to work with chunks of data. Example:

   public function run(StepExecution $stepExecution): void
   {
       $chunk = $stepExecution->getReadData();
       foreach ($chunk as $item) {
           // Process item
       }
       $stepExecution->setProcessedData($processedItems);
   }
   

4. Job Listeners Attach listeners to jobs or steps for logging, validation, or side effects:

   'jobs' => [
       'my_job' => [
           'class' => \App\Jobs\MyJob::class,
           'listeners' => [
               'before' => \App\Listeners\LogJobStart::class,
               'after' => \App\Listeners\SendNotification::class,
           ],
       ],
   ],
   

5. Job Parameters Pass parameters to jobs dynamically:

   $jobRunner->run('my_job', ['file_path' => '/path/to/file.csv']);
   

   Access parameters in the job:

   $filePath = $this->getParameter('file_path');
   

6. Job Dependencies Define dependencies between jobs (e.g., run Job B only after Job A succeeds):

   'jobs' => [
       'job_a' => ['class' => \App\Jobs\JobA::class],
       'job_b' => [
           'class' => \App\Jobs\JobB::class,
           'depends_on' => ['job_a'],
       ],
   ],
   

7. Job Scheduling Use Laravel’s scheduler to run jobs periodically:

   $schedule->job(new \App\Jobs\DailyImportJob())->daily();
   

   Or via the JobRunner:

   $jobRunner->run('daily_import_job');
   

Gotchas and Tips

Pitfalls

1. State Management

   • Issue: StepExecution state (e.g., getReadData(), getProcessedData()) is not automatically persisted between retries. If a job fails and retries, you must ensure data is reprocessed or re-fetched.
   • Fix: Use a database-backed JobRepository (e.g., Akeneo\Batch\Job\JobRepository\DoctrineJobRepository) to persist job state. Configure it in config/batch.php:

     'job_repository' => [
         'class' => \Akeneo\Batch\Job\JobRepository\DoctrineJobRepository::class,
         'entity_manager' => 'default',
     ],
     

2. Chunk Size Mismatch

   • Issue: If a reader produces chunks larger than the configured chunk_size, the job may fail or behave unexpectedly.
   • Fix: Ensure your reader’s next() method respects the chunk size. Example:

     $chunk = [];
     while ($reader->hasNext() && count($chunk) < $stepExecution->getChunkSize()) {
         $chunk[] = $reader->next();
     }
     $stepExecution->addReadData($chunk);
     

3. Circular Dependencies

   • Issue: Defining depends_on in a way that creates circular dependencies (e.g., Job A depends on Job B, which depends on Job A) will cause the job runner to throw an exception.
   • Fix: Use a dependency graph tool or manually validate dependencies before running jobs.

4. Listener Order

   • Issue: Listeners attached to a job or step run in the order they are defined, but this is not always intuitive (e.g., before listeners run in reverse order).
   • Fix: Document listener order explicitly or use named listeners (e.g., before_validate, before_log).

5. Transaction Management

   • Issue: By default, steps do not run in transactions. If a step fails mid-execution, partial data may be written.
   • Fix: Use Akeneo\Batch\Step\TransactionStep to wrap steps in transactions:

     'steps' => [
         'write' => [
             'class' => \Akeneo\Batch\Step\TransactionStep::class,
             'step' => [
                 'class' => \App\Steps\WriteToDatabaseStep::class,
             ],
         ],
     ],
     

6. Reader/Writer Configuration

   • Issue: Readers and writers are not automatically injected into steps. You must manually configure them or use dependency injection.
   • Fix: Use the reader and writer keys in step configuration to pass them in:

     'steps' => [
         'read' => [
             'class' => \App\Steps\ReadFromCsvStep::class,