Mongodb Odm Tailable Cursor Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation Add the bundle to your Laravel project via Composer:

   composer require doctrine/mongodb-odm-tailable-cursor-bundle
   

   Register the bundle in config/app.php under providers:

   Doctrine\Bundle\MongoDBODMTailableCursorBundle\DoctrineMongoDBODMTailableCursorBundle::class,
   

2. Configuration Ensure your config/packages/doctrine_mongodb_odm.yaml includes:

   doctrine_mongodb_odm:
       connections:
           default:
               server: '%env(MONGODB_URL)%'
               options: { }
       document_managers:
           default:
               connection: default
               auto_mapping: true
               tailable_cursor:
                   enabled: true
   

3. First Use Case Create a tailable cursor for a collection (e.g., logs) to stream real-time updates:

   use Doctrine\ODM\MongoDB\DocumentManager;
   use Doctrine\ODM\MongoDB\Query\Builder;
   
   $dm = app(DocumentManager::class);
   $queryBuilder = $dm->createQueryBuilder('App\Document\LogEntry')
       ->field('createdAt')->greaterThan(new \DateTime('-1 hour'));
   
   $cursor = $dm->executeTailableCursor($queryBuilder->getQuery());
   foreach ($cursor as $log) {
       // Process real-time log entry
   }
   

Implementation Patterns

Workflow Integration

1. Real-Time Event Processing Use tailable cursors for streaming logs, notifications, or IoT data:

   $cursor = $dm->executeTailableCursor(
       $dm->createQueryBuilder('App\Document\SensorReading')
           ->field('timestamp')->greaterThan($lastChecked)
   );
   while ($cursor->hasNext()) {
       $reading = $cursor->next();
       event(new SensorReadingProcessed($reading));
   }
   

2. Background Jobs Offload cursor processing to Laravel queues:

   dispatch(new ProcessTailableCursorJob($collection, $query));
   

   Job implementation:

   public function handle() {
       $cursor = $this->dm->executeTailableCursor($this->query);
       while ($cursor->hasNext()) {
           $doc = $cursor->next();
           // Process and sleep to avoid busy-waiting
           sleep(1);
       }
   }
   

3. Hybrid ORM/ODM Combine with Eloquent for mixed data sources:

   // Fetch recent MongoDB docs and sync with SQL
   $recentDocs = $dm->executeTailableCursor($query)->toArray();
   DB::table('sync_logs')->insert($recentDocs);
   

Best Practices

• Avoid Blocking: Use sleep() or usleep() in loops to prevent CPU overload.
• Error Handling: Wrap cursor operations in try-catch for network issues:

  try {
      $cursor = $dm->executeTailableCursor($query);
      while ($cursor->hasNext()) {
          $doc = $cursor->next();
          // Process
      }
  } catch (\MongoDB\Driver\Exception\ConnectionTimeout $e) {
      Log::error("Cursor timeout: {$e->getMessage()}");
  }
  

• Cursor Lifecycle: Always call $cursor->close() when done to free resources.

Gotchas and Tips

Pitfalls

1. Connection Stability

   • Tailable cursors require a persistent connection to MongoDB. Use connection pooling or retry logic for transient failures.
   • Fix: Configure retryWrites: true and retryReads: true in MongoDB connection options.

2. Cursor Timeout

   • MongoDB may kill idle cursors after maxTimeMS (default: 30 minutes).
   • Fix: Implement a heartbeat (e.g., ping()) or re-fetch the cursor periodically.

3. Document Changes

   • If documents are updated/deleted while the cursor is open, behavior depends on MongoDB’s tailable and awaitData flags.
   • Tip: Use awaitData: true to wait for new inserts but skip updates/deletes.

4. Laravel Service Container

   • The DocumentManager is not automatically bound to the container. Manually resolve it:

     $dm = app('doctrine.odm.document_manager');
     

Debugging Tips

• Check Cursor Status:

  if (!$cursor->isDead()) {
      $cursor->rewind(); // Reset cursor
  }
  

• Log Query Performance: Enable MongoDB profiling (db.setProfilingLevel(1)) to monitor slow tailable cursor queries.
• Test Locally: Use mongod --replSet rs0 for replica set testing (tailable cursors behave differently in standalone vs. replica set modes).

Extension Points

1. Custom Cursor Factory Extend Doctrine\ODM\MongoDB\Cursor to add Laravel-specific logic:

   class LaravelTailableCursor extends \Doctrine\ODM\MongoDB\Cursor {
       public function nextWithRetry() {
           try {
               return parent::next();
           } catch (\Exception $e) {
               if ($this->shouldRetry($e)) {
                   return $this->nextWithRetry();
               }
               throw $e;
           }
       }
   }
   

   Register the factory in services.yaml:

   doctrine_mongodb_odm.tailable_cursor.factory: App\Cursor\LaravelTailableCursorFactory
   

2. Event Listeners Hook into cursor events (e.g., postPersist) to trigger side effects:

   $dm->getEventManager()->addEventListener(
       'postPersist',
       function ($event) {
           if ($event->getDocument() instanceof LogEntry) {
               // Notify subscribers
           }
       }
   );
   

3. Laravel Mixins Add cursor methods to Eloquent models via traits:

   trait TailableCursorTrait {
       public function streamRecent($hours = 1) {
           $dm = app('doctrine.odm.document_manager');
           $query = $dm->createQueryBuilder($this->getCollection())
               ->field('createdAt')->greaterThan(new \DateTime("-{$hours} hours"));
           return $dm->executeTailableCursor($query);
       }
   }