Entity Changes Fetcher Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require daimos/entity-changes-fetcher-bundle
   

   Add to config/bundles.php:

   return [
       // ...
       Daimos\EntityChangesFetcherBundle\DaimosEntityChangesFetcherBundle::class => ['all' => true],
   ];
   

2. Configuration Publish the default config:

   php artisan vendor:publish --tag=entity-changes-fetcher-config
   

   Edit config/entity_changes_fetcher.php to define:

   • entities (array of tracked entities)
   • fetch_interval (e.g., 60 for minutes)
   • storage (default: database or cache)

3. First Use Case Track changes for a Post entity:

   // config/entity_changes_fetcher.php
   'entities' => [
       'App\Entity\Post' => [
           'fields' => ['title', 'content', 'publishedAt'],
           'ignore_fields' => ['createdAt', 'updatedAt'],
       ],
   ];
   

   Trigger a manual fetch (for testing):

   php artisan entity-changes:fetch
   

Implementation Patterns

Core Workflow

1. Automatic Fetching Register a console command to run periodically (e.g., via cron):

   * * * * * php artisan entity-changes:fetch >> /dev/null 2>&1
   

   Configure in config/entity_changes_fetcher.php:

   'auto_fetch' => true,
   'fetch_interval' => 60, // minutes
   

2. Event-Driven Integration Listen for change events in your application:

   // src/EventListener/ChangeListener.php
   use Daimos\EntityChangesFetcherBundle\Event\EntityChangedEvent;
   
   class ChangeListener implements EventSubscriber
   {
       public static function getSubscribedEvents()
       {
           return [
               EntityChangedEvent::class => 'onEntityChanged',
           ];
       }
   
       public function onEntityChanged(EntityChangedEvent $event)
       {
           $entity = $event->getEntity();
           $changes = $event->getChanges();
   
           // Process changes (e.g., log, notify, sync)
           logger()->info("Entity {$entity::class} changed:", $changes);
       }
   }
   

3. Storage Strategies

   • Database: Store changes in a entity_changes table (default).

     'storage' => [
         'driver' => 'database',
         'table' => 'entity_changes',
     ],
     

   • Cache: Use Redis/Memcached for transient tracking.

     'storage' => [
         'driver' => 'cache',
         'prefix' => 'entity_changes_',
     ],
     

4. Custom Fetch Logic Extend the fetcher for complex queries:

   // src/Service/PostChangeFetcher.php
   use Daimos\EntityChangesFetcherBundle\Fetcher\AbstractEntityFetcher;
   
   class PostChangeFetcher extends AbstractEntityFetcher
   {
       protected function getQueryBuilder()
       {
           return $this->entityManager
               ->getRepository($this->entityClass)
               ->createQueryBuilder('p')
               ->where('p.publishedAt > :lastFetch')
               ->setParameter('lastFetch', $this->lastFetchTime);
       }
   }
   

   Register in config/entity_changes_fetcher.php:

   'fetchers' => [
       'App\Entity\Post' => App\Service\PostChangeFetcher::class,
   ],
   

Gotchas and Tips

Pitfalls

1. Performance Overhead

   • Avoid tracking large entities or tables with millions of rows.
   • Use ignore_fields to exclude unnecessary columns from change detection.
   • For high-frequency changes, consider cache storage instead of database.

2. Race Conditions

   • If using database storage, ensure your entity_changes table has:

     // migrations/xxxx_create_entity_changes_table.php
     Schema::create('entity_changes', function (Blueprint $table) {
         $table->id();
         $table->string('entity_class');
         $table->json('changes');
         $table->timestamp('fetched_at');
         $table->unique(['entity_class', 'fetched_at']); // Prevent duplicates
     });
     

3. Event Ordering

   • Events fire after the fetch completes, not per-row. For row-level granularity, process changes manually in the fetcher.

4. Doctrine Proxy Issues

   • Disable proxies for tracked entities if changes aren’t detected:

     # config/doctrine.yaml
     orm:
         entity_managers:
             default:
                 metadata_driver:
                     class: Doctrine\ORM\Mapping\Driver\AnnotationDriver
                     # ...
                 dql:
                     string_functions:
                         # ...
                 proxies: false  # Disable proxies for change tracking
     

Debugging Tips

1. Log Fetch Results Enable debug mode in config/entity_changes_fetcher.php:

   'debug' => true,
   

   Logs will appear in storage/logs/laravel.log.

2. Manual Fetch with Verbosity

   php artisan entity-changes:fetch --verbose
   

   Outputs SQL queries and fetched changes.

3. Check Storage

   • Database: Query entity_changes table:

     SELECT * FROM entity_changes ORDER BY fetched_at DESC LIMIT 10;
     

   • Cache: Inspect keys:

     php artisan cache:table
     redis-cli KEYS "entity_changes_*"
     

Extension Points

1. Custom Storage Drivers Implement Daimos\EntityChangesFetcherBundle\Storage\StorageInterface:

   class CustomStorage implements StorageInterface
   {
       public function saveChanges(string $entityClass, array $changes): void
       {
           // Save to external API, S3, etc.
       }
   
       public function getLastFetchTime(string $entityClass): ?\DateTimeInterface
       {
           // Retrieve from custom source
       }
   }
   

   Register in config:

   'storage' => [
       'driver' => App\Storage\CustomStorage::class,
   ],
   

2. Pre/Post-Fetch Hooks Extend the fetcher class and override:

   protected function beforeFetch(): void {}
   protected function afterFetch(): void {}
   

3. Bulk Change Processing For large datasets, batch changes using Doctrine’s BATCH_SIZE:

   $qb->setHint(\Doctrine\ORM\Query::HINT_FORCE_PARTIAL_LOAD, true)
      ->setHint(\Doctrine\ORM\Query::HINT_CACHEABLE, false)
      ->setMaxResults(1000);