Pmk2 Import Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation Add the bundle to your composer.json:

   composer require campusdomar/pmk2-import-bundle
   

   Enable it in config/bundles.php:

   CampusDomar\PuMuKITImportBundle\PuMuKITImportBundle::class => ['all' => true],
   

2. First Use Case: Importing a PuMuKIT XML File

   • Place your exported XML file (e.g., series_export.xml) in a dedicated directory (e.g., public/uploads/imports/).
   • Configure the bundle in config/packages/campusdomar_pumukit_import.yaml:

     campusdomar_pumukit_import:
         import_dir: '%kernel.project_dir%/public/uploads/imports/'
         allowed_channels: ['channel1', 'channel2']  # Optional: Restrict to specific channels
     

   • Run the import command:

     php bin/console campusdomar:pumukit:import series_export.xml
     

   • Verify results in the PuMuKIT admin panel or via API.

Key Files to Review

• Resources/doc/InstallationGuide.md: Prerequisites (PuMuKIT2 setup, database compatibility).
• Resources/doc/ImportExecutionGuide.md: Command-line flags and post-import steps.
• src/Command/ImportCommand.php: Core logic for parsing XML and mapping to PuMuKIT entities.

Implementation Patterns

Workflow: XML-to-PuMuKIT Integration

1. Pre-Import Validation

   • Use the --validate flag to check XML structure before full import:

     php bin/console campusdomar:pumukit:import series_export.xml --validate
     

   • Logs errors to var/log/import_validation.log (configure in config/packages/campusdomar_pumukit_import.yaml).

2. Chunked Imports for Large Files

   • Split XML into smaller files (e.g., by <series> nodes) and import sequentially:

     php bin/console campusdomar:pumukit:import chunk1.xml
     php bin/console campusdomar:pumukit:import chunk2.xml
     

   • Leverage --dry-run to preview changes without committing:

     php bin/console campusdomar:pumukit:import series_export.xml --dry-run
     

3. Post-Import Actions

   • Update Metadata: Use the campusdomar:pumukit:update-metadata command for incremental updates:

     php bin/console campusdomar:pumukit:update-metadata --file=updated_series.xml
     

   • Publish to Channels: Restrict visibility via allowed_channels in config or use the campusdomar:pumukit:publish command:

     php bin/console campusdomar:pumukit:publish --series-id=123 --channel=channel1
     

Integration Tips

• Event Listeners: Extend functionality by subscribing to PuMuKITImportEvents (e.g., ImportPreProcessEvent):

  // src/EventListener/CustomImportListener.php
  namespace App\EventListener;
  
  use CampusDomar\PuMuKITImportBundle\Event\ImportPreProcessEvent;
  use Symfony\Component\EventDispatcher\EventSubscriberInterface;
  
  class CustomImportListener implements EventSubscriberInterface {
      public static function getSubscribedEvents() {
          return [
              ImportPreProcessEvent::class => 'onImportPreProcess',
          ];
      }
  
      public function onImportPreProcess(ImportPreProcessEvent $event) {
          $xmlData = $event->getXmlData();
          // Modify $xmlData before parsing (e.g., add custom attributes)
      }
  }
  

• Custom Mappers: Override default XML-to-entity mapping in config/packages/campusdomar_pumukit_import.yaml:

  campusdomar_pumukit_import:
      custom_mappers:
          series: App\Mapper\CustomSeriesMapper
  

  Implement CampusDomar\PuMuKITImportBundle\Mapper\MapperInterface.

• API Triggers: Automate imports via HTTP requests by exposing a controller:

  // src/Controller/ImportController.php
  use CampusDomar\PuMuKITImportBundle\Command\ImportCommand;
  use Symfony\Component\HttpFoundation\Request;
  
  class ImportController {
      public function import(Request $request, ImportCommand $command) {
          $file = $request->files->get('xml_file');
          $command->run(new ArrayInput(['file' => $file->getPathname()]), new NullOutput());
          return new JsonResponse(['status' => 'import_started']);
      }
  }
  

Gotchas and Tips

Pitfalls

1. XML Schema Mismatches

   • Issue: PuMuKIT v1.7 exports may not align with v2.1 schema (e.g., missing <publication_date>).
   • Fix: Use --strict flag to fail on unknown nodes or patch the XML with defaults:

     <series>
         <publication_date>2023-01-01</publication_date> <!-- Add missing fields -->
     </series>
     

2. Duplicate Entities

   • Issue: Importing the same series twice may cause conflicts (e.g., duplicate series_id).
   • Fix: Use --skip-existing to bypass updates for existing records or implement a custom DuplicateHandler:

     campusdomar_pumukit_import:
         duplicate_handler: App\Handler\MergeDuplicateHandler
     

3. Database Transactions

   • Issue: Large imports may timeout or corrupt data if not wrapped in transactions.
   • Fix: Enable chunked transactions in config:

     campusdomar_pumukit_import:
         transaction_chunk_size: 50  # Commit every 50 entities
     

4. Channel Restrictions

   • Issue: Series may fail to publish if allowed_channels is misconfigured.
   • Fix: Verify channel names match PuMuKIT’s channel table exactly (case-sensitive).

Debugging

• Log Levels: Adjust logging in config/packages/monolog.yaml:

  handlers:
      import:
          type: stream
          path: var/log/import.log
          level: debug  # Set to 'debug' for verbose output
  

• Command Debugging: Use --verbose or --debug flags:

  php bin/console campusdomar:pumukit:import series.xml --debug
  

• SQL Queries: Enable Doctrine profiling in config/packages/dev/doctrine.yaml:

  doctrine:
      dbal:
          profiling: true
  

Extension Points

1. Custom Fields

   • Extend the Series entity to support additional PuMuKIT v2.1 fields (e.g., custom_metadata):

     // src/Entity/SeriesExtension.php
     use Doctrine\ORM\Mapping as ORM;
     
     class SeriesExtension {
         /**
          * @ORM\Column(type="json", nullable=true)
          */
         private $customMetadata;
     }
     

   • Update the mapper to handle the new field:

     public function mapSeries(array $xmlSeries) {
         $series = new Series();
         $series->setCustomMetadata($xmlSeries['custom_field'] ?? null);
         return $series;
     }
     

2. Pre/Post-Import Hooks

   • Implement CampusDomar\PuMuKITImportBundle\Event\ImportEventSubscriber to run logic before/after import:

     public function onImportPostProcess(ImportPostProcessEvent $event) {
         $importedIds = $event->getImportedIds();
         // Send notification or update external systems
     }
     

3. Batch Processing

   • For very large datasets, implement a queue-based approach using Symfony Messenger:

     # config/packages/messenger.yaml
     framework:
         messenger:
             transports:
                 import: '%env(MESSENGER_TRANSPORT_DSN)%'
             routing:
                 'CampusDomar\PuMuKITImportBundle\Message\ImportSeriesMessage': import
     

   • Dispatch messages in the command:

     $bus->dispatch(new ImportSeriesMessage($xmlData, $chunkIndex));
     

Configuration Quirks

• File Paths: Use absolute paths in import_dir to avoid issues with Symfony’s kernel.project_dir.
• Memory Limits: Increase PHP memory for large imports:

  php -d memory_limit=1G bin/console campusdomar:pumukit:import large_file.xml
  

• Time Zones: Ensure publication_date in XML matches the server’s timezone (configure in .env):

  APP_TIMEZONE=UTC