Doctrine Sharding Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require cyber/doctrine-sharding
   

   Add the service provider to config/app.php:

   'providers' => [
       Cyber\DoctrineSharding\DoctrineShardingServiceProvider::class,
   ],
   

2. Configuration Publish the config file:

   php artisan vendor:publish --provider="Cyber\DoctrineSharding\DoctrineShardingServiceProvider"
   

   Edit config/doctrine-sharding.php to define your shards (connections, tables, and routing rules).

3. First Use Case: Querying a Sharded Table

   use Cyber\DoctrineSharding\ShardManager;
   
   $shardManager = app(ShardManager::class);
   $shard = $shardManager->getShardForEntity('App\Models\User', ['id' => 1]);
   $user = $shard->getEntityManager()->find('App\Models\User', 1);
   

Implementation Patterns

Routing Logic

• Consistent Hashing: Use shardBy in config to define routing (e.g., user_id % 3).

  'shards' => [
      'users' => [
          'shard_by' => 'user_id % 3',
          'connections' => ['shard_1', 'shard_2', 'shard_3'],
      ],
  ],
  

• Dynamic Shard Selection: Override routing at runtime:

  $shardManager->setShardForEntity('App\Models\User', 'shard_2');
  

Entity Manager Integration

• Transparent Sharding: Use EntityManager as usual; the package handles shard resolution:

  $user = $entityManager->find('App\Models\User', 1); // Automatically routed
  

• Bulk Operations: For batch inserts/updates, manually select a shard:

  $shard = $shardManager->getShardForEntity('App\Models\Order', ['customer_id' => 123]);
  $shard->getEntityManager()->persist($order);
  $shard->getEntityManager()->flush();
  

Query Building

• Cross-Shard Queries: Avoid them—design queries to target a single shard.
• Shard-Aware DQL: Use SHARD() in DQL for explicit shard selection:

  $query = $entityManager->createQuery(
      'SELECT u FROM App\Models\User u WHERE SHARD(u) = :shard'
  )->setParameter('shard', 'shard_1');
  

Transactions

• Single-Shard Transactions: Use EntityManager transactions normally.
• Multi-Shard Transactions: Not supported—implement compensating transactions or sagas.

Gotchas and Tips

Pitfalls

• No Cross-Shard Joins: Queries spanning shards will fail. Denormalize or replicate data as needed.
• Connection Management: Ensure all shard connections are properly configured in config/database.php.
• Caching: Shard resolution is not cached by default. Cache manually if performance is critical:

  $shardManager->getShardForEntity('App\Models\User', ['id' => 1], true); // Cache hint
  

Debugging

• Enable Logging: Set 'debug' => true in config to log shard resolution.
• Check Shard Resolution: Override ShardResolver to debug routing logic:

  $shardManager->setResolver(new CustomShardResolver());
  

Extension Points

• Custom Resolvers: Implement ShardResolverInterface for complex routing:

  class CustomShardResolver implements ShardResolverInterface {
      public function resolve(string $entityName, array $criteria): string {
          // Custom logic here
      }
  }
  

• Event Listeners: Attach listeners to shard.resolved events for observability:

  event(new ShardResolved($entityName, $shardName, $criteria));
  

Configuration Quirks

• Default Shard: Define a default_shard in config for fallback routing.
• Table Prefixes: Ensure sharded tables use consistent naming conventions (e.g., users_shard_1).

Performance Tips

• Read Replicas: Configure read replicas per shard in config/database.php.
• Connection Pooling: Use pgsql/mysql connection pooling if shards share a database server.