Csv Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require avro/csv-bundle
   

   Enable in config/bundles.php:

   Avro\CsvBundle\AvroCsvBundle::class => ['all' => true],
   Avro\CaseBundle\AvroCaseBundle::class => ['all' => true],
   

2. Basic Configuration (config/packages/avro_csv.yaml):

   avro_csv:
       db_driver: orm
       batch_size: 15
       tmp_upload_dir: "%kernel.root_dir%/../web/uploads/tmp/"
       sample_count: 5
       objects:
           user:
               class: App\Entity\User
               redirect_route: app_user_list
   

3. First Use Case:

   • Add an import link in a Twig template:

     <a href="{{ path('avro_csv_import_upload', {'alias': 'user'}) }}">Import Users</a>
     

   • Navigate to /import/user to upload a CSV and map fields.

Implementation Patterns

Workflows

1. Import Workflow:

   • Upload: User uploads CSV → Bundle validates and shows a preview.
   • Mapping: Auto-detects fields or lets users map columns to entity properties.
   • Execution: Processes in batches (configurable via batch_size) to avoid memory issues.
   • Redirect: Post-import redirects to redirect_route (e.g., a list view).

2. Export Workflow:

   • Navigate to /export/user to trigger a CSV download of all records.
   • Customize queries via events or a dedicated controller (see below).

3. Event-Driven Customization:

   • Row Processing: Listen to RowAddedEvent to modify entities before persistence.

     // src/EventListener/ImportListener.php
     public function setDefaultValues(RowAddedEvent $event) {
         $event->getObject()->setStatus('active');
     }
     

   • Associations: Handle complex relationships via AssociationFieldEvent.

     // src/EventListener/AssociationListener.php
     public function resolveUser(AssociationFieldEvent $event) {
         $user = $this->em->getRepository(User::class)->findOneByEmail($event->getRow()['email']);
         $event->getObject()->setOwner($user);
     }
     

4. Bulk Operations:

   • Use batch_size to control memory usage (default: 15). Increase for large datasets (e.g., batch_size: 100).

Integration Tips

• Validation: Extend the bundle’s form types (AvroCsvBundle\Form\Type\ImportType) to add custom validation.
• Templates: Override Twig templates in templates/AvroCsvBundle/ to customize UI.
• Doctrine Events: Combine with Doctrine lifecycle callbacks (e.g., prePersist) for additional logic.
• Testing: Mock ExporterInterface and ImporterInterface for unit tests:

  $this->container->set('avro_csv.exporter', $this->createMock(ExporterInterface::class));
  

Gotchas and Tips

Pitfalls

1. Memory Limits:

   • Large CSVs (>10K rows) may hit PHP memory limits. Reduce batch_size or use native DB imports for "gargantuan" files (per README).
   • Fix: Increase memory_limit in php.ini or optimize queries.

2. Field Mapping:

   • Case sensitivity in column names can break auto-mapping. Use @ImportExclude for sensitive fields (e.g., passwords).
   • Tip: Use AvroCaseBundle (dependency) to normalize field names.

3. Associations:

   • Default implementation assumes findOneBy on target entities. Custom logic may be needed for complex joins.
   • Debug: Dump $event->getRow() in AssociationFieldEvent to verify data.

4. Temporary Files:

   • Uploads are stored in tmp_upload_dir. Ensure this directory is writable and cleaned up post-import.
   • Tip: Add a cron job to purge old files:

     find /path/to/uploads/tmp/ -type f -mtime +7 -delete
     

5. Symfony 5+ Compatibility:

   • Bundle supports Symfony 5 but may lack autowiring for older services. Use explicit binding:

     # config/services.yaml
     Avro\CsvBundle\Service\Importer: ~
     

Debugging

• Preview Issues: Check sample_count in config to ensure the preview shows correct data.
• Event Not Firing: Verify listeners are registered (e.g., via debug:event-dispatcher).
• CSV Parsing Errors: Validate CSV format (UTF-8 encoding, delimiters). Use tools like CSVLint to pre-check files.

Extension Points

1. Custom Exporters:

   • Extend Avro\CsvBundle\Export\Exporter to add support for non-Doctrine entities (e.g., Elasticsearch models).
   • Example:

     class ElasticExporter implements ExporterInterface {
         public function getContent(): string {
             // Custom logic to fetch from Elasticsearch
             return $this->convertToCsv($data);
         }
     }
     

2. Validation Rules:

   • Add constraints to entity properties (e.g., @Assert\Email) and handle failures in RowAddedEvent:

     public function validateEmail(RowAddedEvent $event) {
         $validator = $this->validator->validate($event->getObject());
         if (count($validator)) {
             throw new \RuntimeException('Validation failed: ' . (string) $validator);
         }
     }
     

3. Performance:

   • For read-heavy exports, cache the CSV generation:

     $cacheKey = 'export_user_' . md5($qb->getQuery()->getSQL());
     if (!$csv = $this->cache->get($cacheKey)) {
         $csv = $exporter->getContent();
         $this->cache->set($cacheKey, $csv, 3600); // Cache for 1 hour
     }
     

4. Security:

   • Restrict import/export routes to roles:

     # config/routes/avro_csv.yaml
     avro_csv_import_upload:
         path: /import/{alias}
         methods: [GET, POST]
         defaults: { _controller: 'avro_csv.import', _role: 'ROLE_ADMIN' }