Message Outbox Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require eventsauce/message-outbox
   

   Ensure eventsauce/eventsauce (v3.0+) and eventsauce/backoff are also installed.

2. Basic Configuration Add the outbox to your Laravel service provider (e.g., AppServiceProvider):

   use EventSauce\MessageOutbox\Outbox;
   use EventSauce\MessageOutbox\OutboxRepository;
   use EventSauce\MessageOutbox\OutboxRepositoryInterface;
   use EventSauce\MessageOutbox\OutboxRepository\Doctrine\DoctrineOutboxRepository;
   use EventSauce\MessageOutbox\OutboxRepository\Doctrine\DoctrineOutboxRepositoryFactory;
   
   public function register()
   {
       $this->app->singleton(OutboxRepositoryInterface::class, function ($app) {
           $entityManager = $app->make(\Doctrine\ORM\EntityManagerInterface::class);
           $factory = new DoctrineOutboxRepositoryFactory();
           return $factory->create($entityManager);
       });
   
       $this->app->singleton(Outbox::class, function ($app) {
           return new Outbox(
               $app->make(OutboxRepositoryInterface::class),
               $app->make(\EventSauce\EventStore::class),
               $app->make(\EventSauce\Middleware\Middleware::class)
           );
       });
   }
   

3. First Use Case: Publishing an Event

   use EventSauce\MessageOutbox\Outbox;
   use EventSauce\MessageOutbox\OutboxMessage;
   
   // In a service or controller
   $outbox = app(Outbox::class);
   
   // Create and dispatch an event
   $event = new \App\Events\UserRegistered('user@example.com');
   $outbox->publish(new OutboxMessage($event));
   
   // The event is now queued for eventual processing.
   

Implementation Patterns

Workflow: Eventual Consistency via Outbox

1. Transaction Boundaries Wrap event publishing and database operations in a single transaction to ensure consistency:

   DB::transaction(function () use ($outbox, $user) {
       $user->save();
       $outbox->publish(new OutboxMessage(new UserRegistered($user->id)));
   });
   

2. Middleware Integration Use EventSauce middleware to enrich events before publishing:

   $middleware = new \EventSauce\Middleware\Middleware();
   $middleware->add(new \App\Middleware\AddMetadata());
   
   $outbox = new Outbox($repository, $eventStore, $middleware);
   

3. Polling the Outbox Implement a Laravel command to process outbox messages (e.g., via a queue worker):

   use EventSauce\MessageOutbox\OutboxRepositoryInterface;
   use EventSauce\MessageOutbox\OutboxMessage;
   
   class ProcessOutboxMessages implements ShouldQueue
   {
       protected $outboxRepo;
       protected $eventStore;
   
       public function __construct(OutboxRepositoryInterface $outboxRepo, \EventSauce\EventStore $eventStore)
       {
           $this->outboxRepo = $outboxRepo;
           $this->eventStore = $eventStore;
       }
   
       public function handle()
       {
           $messages = $this->outboxRepo->getMessagesToPublish();
           foreach ($messages as $message) {
               $this->eventStore->append($message->event());
               $this->outboxRepo->markAsPublished($message->id());
           }
       }
   }
   

4. Event Deduplication Use the outbox’s idempotent() flag to avoid reprocessing:

   $outbox->publish(new OutboxMessage($event, idempotent: true, id: 'user-registered-123'));
   

Gotchas and Tips

Pitfalls

1. Transaction Isolation

   • If the outbox poller runs in a separate process, ensure it uses READ_COMMITTED isolation to avoid missing messages due to uncommitted transactions.
   • Fix: Configure Doctrine:

     $entityManager->getConnection()->setTransactionIsolation(\PDO::TRANSACTION_READ_COMMITTED);
     

2. Message Ordering

   • The outbox does not guarantee FIFO ordering across restarts. Use event timestamps or sequence numbers if order matters.
   • Workaround: Add a published_at column and sort by it.

3. Doctrine Schema Mismatch

   • The package expects a specific schema. Run migrations before using the outbox:

     php artisan doctrine:migrations:execute --up
     

   • Tip: Extend DoctrineOutboxRepository to customize the schema if needed.

4. Event Sauce Version Lock

   • The package is tightly coupled to eventsauce/eventsauce:^3.0. Upgrading EventSauce may break compatibility.
   • Tip: Test thoroughly after major version updates.

Debugging

1. Stuck Messages

   • Check for locked rows in the outbox_message table:

     SELECT * FROM outbox_message WHERE status = 'pending' AND locked_at IS NOT NULL;
     

   • Fix: Manually update locked_at to NULL or restart the poller.

2. Missing Events

   • Verify the poller is running and the messages_to_publish query returns results:

     $this->outboxRepo->getMessagesToPublish(); // Should return OutboxMessage[]
     

Extension Points

1. Custom Outbox Repository Implement OutboxRepositoryInterface for non-Doctrine databases (e.g., Eloquent):

   class EloquentOutboxRepository implements OutboxRepositoryInterface
   {
       // Implement getMessagesToPublish(), markAsPublished(), etc.
   }
   

2. Event Filtering Override getMessagesToPublish() to filter events dynamically:

   class FilteredOutboxRepository implements OutboxRepositoryInterface
   {
       public function getMessagesToPublish(): array
       {
           return $this->outboxRepo->getMessagesToPublish()
               ->where('event_type', '!=', 'App\Events\LogEntryCreated')
               ->get();
       }
   }
   

3. Retry Logic Use eventsauce/backoff to customize retry delays for failed events:

   $backoff = new \EventSauce\Backoff\ExponentialBackoff(100, 3);
   $outbox = new Outbox($repo, $eventStore, $middleware, $backoff);
   

4. Laravel Queue Integration Dispatch the poller as a queued job for async processing:

   ProcessOutboxMessages::dispatch()->onQueue('outbox');