Advanced Csv Connector Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation:

   • Add the repository to composer.json under repositories:

     "repositories": [
         {
             "type": "vcs",
             "url": "https://github.com/ClickAndMortar/CustomEntityBundle"
         }
     ]
     

   • Require the bundle:

     composer require clickandmortar/advanced-csv-connector-bundle
     

   • Enable the bundle in config/bundles.php:

     return [
         // ...
         ClickAndMortar\AdvancedCsvConnectorBundle\ClickAndMortarAdvancedCsvConnectorBundle::class => ['all' => true],
     ];
     

2. First Use Case:

   • Override the default CSV mapping for an entity (e.g., Product or Family) by creating a custom mapping file in:

     config/akeneo/advanced-csv-connector/mapping/[entity_name].php
     

   • Example structure:

     return [
         'columns' => [
             'identifier' => 'sku',
             'families' => 'family',
             'enabled' => 'active',
             // Custom mappings...
         ],
         'headers' => [
             'sku' => 'SKU',
             'family' => 'Family',
             'active' => 'Is Active',
         ],
     ];
     

3. Trigger a Test Export:

   • Use the Akeneo CLI to test your mapping:

     php bin/console akeneo:csv-export --entity=Product --file=test_export.csv
     

Implementation Patterns

Workflow: Custom CSV Mapping

1. Define Mappings:

   • Create a mapping file for each entity (e.g., Product, Family, Category).
   • Use the columns key to map CSV headers to Akeneo entity properties.
   • Use the headers key to define the CSV header row (optional if using default headers).

2. Dynamic Value Transformation (Lua):

   • Enable Lua scripting for dynamic value updates during import/export:

     # config/packages/click_and_mortar_advanced_csv_connector.yaml
     click_and_mortar_advanced_csv_connector:
         lua_enabled: true
     

   • Example Lua script for a custom column (e.g., price):

     -- In your mapping file or a separate Lua script
     function transformPrice(value)
         return tonumber(value) * 1.1 -- Apply 10% tax
     end
     

   • Reference the Lua function in your mapping:

     'price' => [
         'header' => 'Price',
         'transformer' => 'transformPrice',
     ],
     

3. Integration with Akeneo Jobs:

   • Extend Akeneo’s job system to use custom mappings:

     # config/akeneo/job_instance/[custom_job].yaml
     job_instance:
         code: custom_product_import
         type: import
         connector: advanced_csv
         processor: custom_product_processor
     

   • Create a custom processor to handle advanced logic:

     // src/Akeneo/Pim/Enrichment/Processor/CustomProductProcessor.php
     use ClickAndMortar\AdvancedCsvConnectorBundle\Processor\AdvancedCsvProcessor;
     
     class CustomProductProcessor extends AdvancedCsvProcessor
     {
         public function process($data)
         {
             // Custom logic before/after parent processing
             $this->applyCustomRules($data);
             return parent::process($data);
         }
     }
     

4. Validation and Error Handling:

   • Override validation rules in your mapping:

     'columns' => [
         'sku' => [
             'header' => 'SKU',
             'validator' => 'required|min:3|max:20',
         ],
     ],
     

   • Handle errors gracefully by extending the error handler:

     // src/Akeneo/Pim/Enrichment/ErrorHandler/CustomErrorHandler.php
     use ClickAndMortar\AdvancedCsvConnectorBundle\ErrorHandler\AdvancedCsvErrorHandler;
     
     class CustomErrorHandler extends AdvancedCsvErrorHandler
     {
         public function handleError($error, $context)
         {
             // Custom error logging or notification
             $this->logError($error);
             return parent::handleError($error, $context);
         }
     }
     

Common Patterns

Gotchas and Tips

Pitfalls

1. Mapping File Overrides:

   • Issue: Custom mappings in config/ may be overwritten during updates.
   • Fix: Use the config_loader service to merge mappings dynamically:

     # config/packages/click_and_mortar_advanced_csv_connector.yaml
     click_and_mortar_advanced_csv_connector:
         mapping_loader:
             - '%kernel.project_dir%/config/akeneo/advanced-csv-connector/mapping'
     

2. Lua Security Restrictions:

   • Issue: Lua scripts are sandboxed; restricted functions may cause errors.
   • Fix: Test scripts in a safe environment first. Use PHP transformers for complex logic:

     'price' => [
         'header' => 'Price',
         'transformer' => 'App\Transformer\PriceTransformer',
     ];
     

3. Performance with Large Files:

   • Issue: Lua scripts or complex PHP transformers slow down processing.
   • Fix: Batch processing or use Akeneo’s chunked import:

     php bin/console akeneo:csv-import --entity=Product --file=large_file.csv --chunk-size=100
     

4. Header Mismatches:

   • Issue: CSV headers don’t match Akeneo’s expected format.
   • Fix: Use the headers key to explicitly define the CSV header row:

     'headers' => [
         'custom_sku' => 'SKU',
         'custom_family' => 'Family',
     ],
     

5. Dependency Conflicts:

   • Issue: Conflicts with other CSV-related bundles (e.g., akeneo/connect).
   • Fix: Check composer.json for version constraints and use replace:

     "replace": {
         "akeneo/connect": "v1.0.*"
     }
     

Debugging Tips

1. Enable Verbose Logging:

   • Add to config/packages/monolog.yaml:

     handlers:
         advanced_csv:
             type: stream
             path: "%kernel.logs_dir%/advanced_csv.log"
             level: debug
             channels: ["akeneo"]
     

2. Validate Mappings:

   • Use the akeneo:csv-validate command to check mappings before import/export:

     php bin/console akeneo:csv-validate --entity=Product --file=test.csv
     

3. Inspect Processed Data:

   • Override the processor to dump data for debugging:

     public function process($data)
     {
         file_put_contents(
             '/tmp/debug.csv',
             print_r($data, true),
             FILE_APPEND
         );
         return parent::process($data);
     }
     

Extension Points

1. Custom Transformers:

   • Create a service for reusable transformers:

     # config/services.yaml
     services:
         App\Transformer\CustomTransformer:
             tags: ['akeneo.csv.transformer']
     

2. Event Listeners:

   • Listen to akeneo_csv.pre_process or akeneo_csv.post_process events:

     // src/EventListener/CsvListener.php
     use ClickAndMortar\AdvancedCsvConnectorBundle\Event\CsvEvent;
     
     class CsvListener
     {
         public function onPreProcess(CsvEvent $event)
         {
             $event->setData($this->modifyData($event->getData()));
         }
     }
     

     Register the listener in services.yaml:

     tags: ['kernel.event_listener', { event: 'akeneo_csv.pre_process', method: 'onPreProcess' }]
     

3. Custom Job Types:

   • Extend the bundle to support custom job types (e.g., export_to_s3):

     // src/