Data Generator Bundle Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require akeneo-labs/data-generator-bundle:~0.3
   

   Register the bundle in app/AppKernel.php:

   $bundles[] = new Pim\Bundle\DataGeneratorBundle\PimDataGeneratorBundle();
   

2. Prerequisites: Ensure Akeneo PIM (v1.3–1.6) is installed with channels, locales, and currencies preconfigured.

3. First Use Case: Generate base fixtures (attributes, families) and product CSV files separately:

   # Step 1: Generate fixtures (e.g., attributes, families)
   php bin/console pim:generate:fixtures config/fixtures.yaml
   
   # Step 2: Generate product CSV (requires fixtures to exist)
   php bin/console pim:generate:products-file config/products.yaml
   

4. Configuration Files: Start with the example YAML templates in Resources/examples.

Implementation Patterns

Workflows

1. Two-Phase Generation:

   • Fixtures First: Generate attributes, families, and categories before products.

     # config/fixtures.yaml
     data_generator:
       output_dir: /tmp/fixtures/
       entities:
         attributes:
           count: 100
           identifier_attribute: "sku"
         families:
           count: 20
           attributes_count: 50
     

   • Products Second: Use the generated fixtures to create product CSVs.

     # config/products.yaml
     data_generator:
       output_dir: /tmp/products/
       entities:
         products:
           count: 5000
           mandatory_attributes: [sku, name]
           force_values: { brand: "TestBrand" }
     

2. Integration with Fixtures:

   • Copy generated fixture files into Akeneo’s app/import/fixtures/ and update installer_data in config/config.yml:

     installer_data:
       attributes: ["/tmp/fixtures/attributes.csv"]
       families: ["/tmp/fixtures/families.csv"]
     

3. Automated Testing:

   • Use the bundle in PHPUnit tests to spin up a fresh PIM instance with synthetic data:

     // tests/Functional/DataGeneratorTest.php
     public function testProductGeneration()
     {
         $this->generateFixtures('config/fixtures.yaml');
         $this->generateProducts('config/products.yaml');
         $this->assertFileExists('/tmp/products/products.csv');
     }
     

4. CI/CD Pipelines:

   • Generate data in pre-deployment scripts to populate staging environments:

     # .github/workflows/deploy.yml
     - name: Generate test data
       run: |
         php bin/console pim:generate:fixtures config/fixtures.yaml
         php bin/console pim:generate:products-file config/products.yaml
     

Key Patterns

• Faker Integration: Leverage Faker’s providers (e.g., name, text, address) for realistic data:

  force_values:
    description: "{{ sentence(5) }}"  # Faker syntax
  

• Bulk Operations: Use filled_attributes_count and categories_count to simulate real-world complexity.
• Delimiter Customization: Override CSV delimiters for non-standard imports:

  delimiter: ";"
  

Gotchas and Tips

Pitfalls

1. Order Dependency:

   • Error: Command "pim:generate:products-file" failed: Missing attribute "sku".
   • Fix: Always run pim:generate:fixtures first to populate attributes/families.

2. Output Directory Permissions:

   • Error: Failed to write to /tmp/fixtures/attributes.csv: Permission denied.
   • Fix: Ensure the output_dir is writable:

     chmod -R 777 /tmp/fixtures
     

3. Faker Version Conflicts:

   • Error: Class 'Faker\Provider\Base' not found.
   • Fix: Pin Faker to ~1.4.0 in composer.json to match the bundle’s requirement.

4. Locale/Channel Mismatches:

   • Error: Generated products lack locale-specific values (e.g., name_en_US).
   • Fix: Explicitly define locales in the config or pre-generate translations:

     force_values:
       name_en_US: "Product {{ randomNumber(1000) }}"
     

Debugging Tips

• Validate YAML: Use yaml-lint or php bin/console debug:config data_generator to check syntax.
• Log Generation: Enable debug mode to see Faker’s random value generation:

  php bin/console pim:generate:products-file config/products.yaml --env=dev
  

• Dry Runs: Test with small counts first (e.g., count: 10) before scaling to 10,000+ products.

Extension Points

1. Custom Faker Providers: Extend Faker to add domain-specific data (e.g., ProductNameProvider):

   // src/Akeneo/DataGeneratorBundle/Faker/ProductNameProvider.php
   class ProductNameProvider extends \Faker\Provider\Base
   {
       public function productName() { ... }
   }
   

   Register it in config/services.yaml:

   services:
     Faker\Generator:
       calls:
         - [addProvider, ['@akeneo.data_generator.faker.product_name']]
   

2. Post-Generation Hooks: Use Symfony’s event system to process generated files:

   // src/EventListener/DataGeneratorListener.php
   class DataGeneratorListener implements KernelEvents
   {
       public function onKernelTerminate(KernelEvents::TERMINATE, Event $event)
       {
           if ($event->getRequest()->get('_route') === 'pim_generate_products_file') {
               $this->postProcess('/tmp/products/products.csv');
           }
       }
   }
   

3. Parallel Generation: For large datasets, split product generation into chunks using a loop:

   for i in {1..10}; do
     php bin/console pim:generate:products-file config/products_${i}.yaml
   done
   

Configuration Quirks

• start_index: Useful for paginated imports (e.g., start_index: 1000 for batch 2).
• force_values vs. mandatory_attributes:
  • force_values: Overrides any occurrence of the attribute.
  • mandatory_attributes: Ensures the attribute is always filled (but with random data).
• Empty Families: If families.attributes_count: 0, products may fail validation. Set a minimum (e.g., 2).