Import Bundle Laravel Package

Getting Started

Minimal Steps

1. Installation Add the package via Composer:

   composer require bigfoot/import-bundle:dev-master
   

   Register the bundle in app/AppKernel.php:

   new BigFoot\ImportBundle\BigFootImportBundle(),
   

2. Generate Base Structure Run the update command to scaffold the bundle:

   php app/console generate:update
   

   Generate your entity from a CSV/XML file:

   php app/console generate:doctrine:entity
   

   Update the database schema:

   php app/console doctrine:schema:update --force
   

3. First Use Case: CSV Import

   • Create a Services directory in your bundle.
   • Extend AbstractSimpleDataMapper for your entity:

     namespace AppBundle\Services;
     use BigFoot\ImportBundle\Services\AbstractSimpleDataMapper;
     
     class MyEntityDataMapper extends AbstractSimpleDataMapper
     {
         const FIELD_ID = 'id';
         const FIELD_NAME = 'name';
         const FIELD_DESCRIPTION = 'description';
     
         protected function getEntityClass()
         {
             return 'AppBundle\Entity\MyEntity';
         }
     }
     

   • Use the mapper in a controller/service to import data:

     $mapper = new MyEntityDataMapper();
     $mapper->import($filePath);
     

Implementation Patterns

Workflows

1. CSV Import Workflow

   • Pre-Processing: Validate file structure (headers, delimiters) before mapping.
   • Mapping: Define constants for CSV headers and override mapData() in AbstractSimpleDataMapper:

     protected function mapData($data)
     {
         $entity = new $this->getEntityClass();
         $entity->setId($data[self::FIELD_ID]);
         $entity->setName($data[self::FIELD_NAME]);
         return $entity;
     }
     

   • Post-Processing: Handle duplicates, errors, or validation via postImport():

     protected function postImport($entities)
     {
         $em = $this->getEntityManager();
         foreach ($entities as $entity) {
             $em->persist($entity);
         }
         $em->flush();
     }
     

2. XML Import Workflow

   • Use XmlMapper (documented in ./Resources/doc/xmlmapper.md) for XML-specific logic.
   • Example: Map XML nodes to entity properties:

     $xmlMapper = new \BigFoot\ImportBundle\Services\XmlMapper();
     $xmlMapper->map($xmlFile, $entity);
     

3. Batch Processing

   • For large files, implement chunked imports:

     $mapper->setChunkSize(100); // Process 100 rows at a time
     $mapper->import($filePath);
     

Integration Tips

• Event Listeners: Attach listeners to import.pre/import.post events for cross-cutting concerns (e.g., logging, notifications).
• Validation: Use Symfony’s Validator component to validate imported data before persistence.
• Dependency Injection: Register mappers as services in services.yml for reusable imports:

  services:
      app.my_entity_mapper:
          class: AppBundle\Services\MyEntityDataMapper
          arguments: ['@doctrine.orm.entity_manager']
  

Gotchas and Tips

Pitfalls

1. Deprecated Package

   • Last release in 2014; expect compatibility issues with modern Laravel/Symfony (e.g., Doctrine ORM, PHP 8.x).
   • Mitigation: Fork the repo and update dependencies (e.g., doctrine/orm, symfony/console) manually.

2. CSV Parsing Quirks

   • Assumes strict header matching (case-sensitive). Use strtolower() in constants if headers vary:

     const FIELD_NAME = strtolower('Name'); // 'name' vs 'NAME'
     

   • Delimiters: Defaults to comma (,). Override getDelimiter() if using tabs (\t) or pipes (|).

3. XML Namespace Issues

   • XML imports may fail if namespaces aren’t handled. Extend XmlMapper or pre-process XML:

     $xml = simplexml_load_file($filePath);
     $xml->registerXPathNamespace('ns', 'http://example.com/ns');
     

4. Entity Manager Injection

   • AbstractSimpleDataMapper expects an EntityManager via constructor. Inject it explicitly:

     $mapper = new MyEntityDataMapper($em);
     

Debugging

• Log Imports: Override preImport() to log file paths/headers:

  protected function preImport($filePath)
  {
      $this->logger->info('Importing: ' . $filePath);
      $headers = $this->getCsvHeaders($filePath);
      $this->logger->debug('Headers:', $headers);
  }
  

• Validate Data: Use var_dump() or dd() in mapData() to inspect raw input:

  protected function mapData($data)
  {
      dd($data); // Debug raw CSV row
      // ...
  }
  

• Check for Duplicates: Implement prePersist() to avoid conflicts:

  protected function prePersist($entity)
  {
      if ($this->entityExists($entity)) {
          throw new \RuntimeException('Duplicate entity detected');
      }
  }
  

Extension Points

1. Custom Mappers

   • Extend AbstractSimpleDataMapper for complex logic (e.g., nested entities, multi-step imports).
   • Example: Import related entities in a single pass:

     class OrderDataMapper extends AbstractSimpleDataMapper
     {
         protected function mapData($data)
         {
             $order = new Order();
             $order->setCustomer($this->mapCustomer($data));
             return $order;
         }
     
         private function mapCustomer($data)
         {
             $customer = new Customer();
             $customer->setEmail($data[self::FIELD_CUSTOMER_EMAIL]);
             return $customer;
         }
     }
     

2. File Format Support

   • Add JSON/Excel support by extending the core logic. Example for JSON:

     $data = json_decode(file_get_contents($filePath), true);
     $mapper->import($data); // Modify mapper to handle arrays
     

3. Configuration

   • Override default settings (e.g., chunk size, delimiter) via bundle configuration:

     # app/config/config.yml
     bigfoot_import:
           default_chunk_size: 500
           default_delimiter: ';'
     

   • Access config in mapper:

     $chunkSize = $this->container->getParameter('bigfoot_import.default_chunk_size');