Import Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require dmytrof/import-bundle
   

   Enable in config/bundles.php:

   Dmytrof\ImportBundle\DmytrofImportBundle::class => ['all' => true],
   

2. First Use Case: Create a CSV file (data.csv) with headers matching your entity fields (e.g., name,email,role). Define an importer class (e.g., UserImporter) extending Dmytrof\ImportBundle\Importer\AbstractImporter:

   namespace App\Importer;
   
   use App\Entity\User;
   use Dmytrof\ImportBundle\Importer\AbstractImporter;
   
   class UserImporter extends AbstractImporter
   {
       protected $entityClass = User::class;
       protected $fields = ['name', 'email', 'role'];
   
       public function transform(array $data): array
       {
           return [
               'name' => $data['name'],
               'email' => strtolower($data['email']),
               'roles' => ['ROLE_' . strtoupper($data['role'])],
           ];
       }
   }
   

3. Run Import: Use the CLI command:

   php bin/console dmytrof:import:run App\Importer\UserImporter data.csv
   

Implementation Patterns

Core Workflow

1. Define Importers: Extend AbstractImporter for each entity type. Key methods:

   • getEntityClass(): Return the target entity class.
   • getFields(): Define CSV headers → entity properties mapping.
   • transform(array $data): Sanitize/convert raw data (e.g., normalize emails, map roles).

2. Batch Processing: Configure chunk size in config/packages/dmytrof_import.yaml:

   dmytrof_import:
       chunk_size: 50  # Process 50 rows per transaction
   

3. Validation: Override validate(array $data) to enforce rules (e.g., email format):

   public function validate(array $data): void
   {
       if (!filter_var($data['email'], FILTER_VALIDATE_EMAIL)) {
           throw new \RuntimeException('Invalid email: ' . $data['email']);
       }
   }
   

4. Post-Import Actions: Use postImport() to trigger events (e.g., send welcome emails):

   public function postImport(array $entities): void
   {
       foreach ($entities as $user) {
           $this->mailer->send(new WelcomeEmail($user));
       }
   }
   

Integration Tips

• Doctrine Events: Hook into prePersist/preUpdate for additional logic.
• Symfony Forms: Use the bundle’s ImportType to build admin forms for file uploads.
• Testing: Mock AbstractImporter in PHPUnit:

  $importer = $this->createMock(UserImporter::class);
  $importer->method('transform')->willReturn(['name' => 'Test']);
  

Gotchas and Tips

Pitfalls

1. Field Mismatches:

   • Ensure CSV headers exactly match getFields() (case-sensitive).
   • Fix: Use trim() in transform() to handle whitespace.

2. Transaction Limits:

   • Large imports may hit Doctrine’s transaction size limits.
   • Fix: Reduce chunk_size or use flush() manually in transform().

3. Circular References:

   • Importing entities with self-references (e.g., User → Profile) may fail.
   • Fix: Use setEntityManager() to bypass Doctrine’s proxy detection:

     $this->setEntityManager($entityManager);
     

4. Memory Leaks:

   • Loading all entities into memory during validation.
   • Fix: Stream data with SplFileObject or use validate() sparingly.

Debugging

• Log Raw Data: Dump $this->getData() in transform() to inspect input:

  file_put_contents('debug.csv', print_r($this->getData(), true));
  

• Symfony Profiler: Enable dmytrof_import.debug in config to track import stats.

Extension Points

1. Custom Writers: Override writeEntity() to support non-Doctrine storage (e.g., Elasticsearch):

   protected function writeEntity($entity): void
   {
       $this->elasticsearch->index($entity);
   }
   

2. Predefined Mappers: Use Dmytrof\ImportBundle\Mapper\ArrayMapper for dynamic field mapping:

   $mapper = new ArrayMapper($this->getEntityClass());
   $mapper->map($data, $entity);
   

3. Event Dispatching: Subscribe to dmytrof.import.pre/post events in services:

   # config/services.yaml
   App\EventListener\ImportListener:
       tags:
           - { name: kernel.event_listener, event: dmytrof.import.pre, method: onPreImport }