Doctrine Shard Manager Bundle for Symfony

Doctrine Shard Manager Bundle is a Symfony bundle for horizontal data sharding on top of Doctrine ORM and DBAL. The bundle provides transparent routing of Connection and EntityManager per shard, pluggable resolution strategies (hash, range, uuidv7), a PSR-6 cache layer, and a CLI for greenfield provisioning and additive migrations.

Key Features

• Transparent sharding. Callers write ShardedRepository calls or ShardContext::withShard() / withEntity() callbacks; the bundle selects the right Connection and EntityManager for the entity being touched.
• Pluggable strategies. Three concrete strategies ship out of the box — hash (SHA-256 modulo), range (binary search over a range_table partition), uuid (UUIDv7 with embedded shard index). Custom strategies register via the app.shard_strategy tag (see ADR-0002).
• Dual configuration surface. PHP #[Sharding] attribute on the entity class, OR YAML under doctrine_shard.sharding_configs.<FQCN>. Resolution precedence is YAML > attribute (see docs/architecture.md::I-RES1).
• Cache-first resolution. Optional PSR-6 CacheItemPoolInterface per strategy and per resolver; cache failures degrade gracefully (logged + bypassed, never propagated).
• Cross-shard reads. ShardedFinder::find(callable $query, array $shardIds) fans out a read query to multiple shards lazily via \Generator (see ADR-0004).
• Schema bootstrap. bin/console shard:add <id> provisions a shard's schema from filtered entity metadata. bin/console shard:migrate <id> runs Doctrine Migrations on a single named shard and is additive (see ADR-0003).
• Doctrine 3.3 + DBAL 4.2 compatibility, PHP 8.3 strict types.

Architecture

             +--------------------------+
             |   Symfony Application    |
             |     (consumer code)      |
             +------------+-------------+
                          |
               uses interfaces in src/Contract/
                          |
             +------------v-------------+
             |  ShardContext /          |
             |  ShardedRepository       |
             |  ShardedFinder (read)    |
             +------------+-------------+
                          |
             +------------v-------------+
             |   ShardResolver          |   <-- strategy selection, key extraction
             +------------+-------------+
                          |
             +------------v-------------+      +-----------------------+
             |   ShardStrategy (IF)    | <--> | Hash / Range / UUID   |
             +--------------------------+      +-----------------------+
                          |
             +------------v-------------+
             |  ShardConnectionManager  |   <-- DBAL Connection per shard
             +------------+-------------+
                          |
             +------------v-------------+
             |  EntityManagerProxy      |   <-- ORM EntityManager per shard
             +------------+-------------+
                          |
             +------------v-------------+
             |  Doctrine DBAL / ORM     |
             +--------------------------+

Layering rules (see docs/architecture.md for the full invariant list):

• Contract/* interfaces are the only types consumers depend on directly. All concrete implementations are un-tagged internal; consumers MUST type-hint against the interfaces.
• Strategy/* depends only on Contract/ShardStrategyInterface and Config/Dto/*.
• Resolver/* depends only on Contract/*, Attribute/*, and Strategy/* instances injected at construction.
• Context/* orchestrates Resolver, ConnectionManager, EntityManagerProxy — it does not touch Doctrine directly.
• Manager/* is a thin Doctrine binding layer (ShardConnectionManager wraps Connection lookup; EntityManagerProxy wraps EntityManager lookup).
• Repository/* provides a higher-level facade; it composes ShardContext and ShardResolver.

Key interfaces

Strategy contracts

• ShardStrategyInterface — resolveShardId(mixed $key, array $options = []), getTotalShards(), resolveShardIndex(mixed $key), getCacheKey(mixed $key). The single seam for pluggable strategies.
• AbstractShardStrategy — base class that exposes three protected helpers (normalizeKey, defaultCacheKey, resolveShardIndexFromId). Override public methods on the abstract class for custom strategies; concrete strategies (HashShardStrategy, RangeShardStrategy, UuidShardStrategy) are reference implementations and may change between minor versions.

Resolver and context

• ShardResolverInterface — resolveShardId(object $entity), resolveShardIdFromId(mixed $id, string $entityClass), resolveShardIdFromCriteria(array $criteria, string $entityClass), getShardingConfig(string $entityClass). Merges YAML and #[Sharding] attribute.
• ShardContextInterface — withShard(string $shardId, callable $cb), withEntity(object $entity, callable $cb), getCurrentShardId(), getAvailableShards(). The state machine that switches Connection and EntityManager for the duration of the callback.

Manager layer

• ShardConnectionManagerInterface — switchToShard(string $shardId), getCurrentConnection(), getConnectionForShard(string $shardId). Owns the shard-to-Connection registry.
• EntityManagerProxyInterface — forShard(string $shardId), getAvailableShards(). Owns the shard-to-EntityManager registry.

Repository and finder facades

• ShardedRepositoryInterface — findOneById, findBy, findOneBy, persistToShard, removeFromShard, flushShard. Composes ShardContext and ShardResolver; business code stays unchanged.
• ShardedFinder (concrete) — find(callable $query, list<string> $shardIds): \Generator. Bypasses the write-path ShardContext; throws on the first shard failure with the per-shard context framed in the exception message.

Installation

Run the following command to install the bundle:

composer require elriseio/doctrine-shard-manager-bundle

Register the bundle in config/bundles.php (if you are not using Symfony Flex):

Elrise\Bundle\DoctrineShardManager\ElriseDoctrineShardBundle::class => ['all' => true],

The bundle reads its configuration from config/packages/doctrine_shard.yaml (see Configuration below) and registers all six public services automatically. No additional services.yaml wiring is required for the default strategies.

Configuration

config/packages/doctrine_shard.yaml example for a four-shard hash setup keyed by userId:

doctrine_shard:
  cache_ttl: 86400

  shard:
    hash:
      total_shards: 4
      shard_index_offset: 0
      shard_prefix: 'shard_'
    range:
      shard_prefix: 'shard_'
      shard_index_offset: 0
      range_table:
        - { min: 0,   max: 999_999,    shard: 'shard_0' }
        - { min: 1_000_000, max: 1_999_999, shard: 'shard_1' }
    uuid:
      total_shards: 16
      shard_index_offset: 0
      shard_prefix: 'shard_'
      shard_bits: 4

  sharding_configs:
    App\Entity\User:
      strategy: hash
      key: userId
      shardCount: 4
    App\Entity\Order:
      strategy: uuid
      key: id
      shardCount: 16

  resolver:
    cache:
      enabled: true
      pool_service_id: cache.app

  connections:
    shard_0: '@doctrine.dbal.default_shard_0_connection'
    shard_1: '@doctrine.dbal.default_shard_1_connection'
    shard_2: '@doctrine.dbal.default_shard_2_connection'
    shard_3: '@doctrine.dbal.default_shard_3_connection'

  entity_managers:
    shard_0: '@doctrine.orm.default_shard_0_entity_manager'
    shard_1: '@doctrine.orm.default_shard_1_entity_manager'
    shard_2: '@doctrine.orm.default_shard_2_entity_manager'
    shard_3: '@doctrine.orm.default_shard_3_entity_manager'

doctrine.dbal.default_shard_<N>_connection and doctrine.orm.default_shard_<N>_entity_manager are the standard Doctrine connection / EM service IDs that Symfony's doctrine bundle exposes when you declare dbal: { connections: { default_shard_0: ~, default_shard_1: ~, ... } } and orm: { entity_managers: { default_shard_0: ~, default_shard_1: ~, ... } } in config/packages/doctrine.yaml.

The #[Sharding] attribute (alternative to YAML)

use Elrise\Bundle\DoctrineShardManager\Attribute\Sharding;

#[Sharding(
    strategy: 'hash',
    key: 'userId',
    shardCount: 4,
)]
class User
{
    // ...
}

The key parameter may also be string[] for composite keys:

#[Sharding(
    strategy: 'hash',
    key: ['tenantId', 'userId'],
    shardCount: 4,
)]
class User
{
    // ...
}

YAML sharding_configs.<FQCN> overrides the attribute when both are present.

Usage

Through ShardContext

use Elrise\Bundle\DoctrineShardManager\Context\ShardContext;

final class UserService
{
    public function __construct(private ShardContext $shardContext) {}

    public function loadFromShard(int $userId): ?User
    {
        return $this->shardContext->withEntity(
            $this->buildStubUser($userId),
            function (EntityManagerInterface $em, string $shardId) use ($userId): ?User {
                return $em->getRepository(User::class)->find($userId);
            },
        );
    }
}

withShard($shardId, fn) is the explicit variant when the caller already knows the target shard (e.g. a cron job iterating every shard).

Through ShardedRepository

use Elrise\Bundle\DoctrineShardManager\Repository\ShardedRepository;

final class UserController
{
    public function __construct(private ShardedRepository $userRepo) {}

    public function show(int $userId): Response
    {
        $user = $this->userRepo->findOneById($userId, User::class);

        if (null === $user) {
            throw new NotFoundHttpException();
        }

        return new Response(sprintf('Hello, %s!', $user->getDisplayName()));
    }
}

ShardedRepository::findBy, findOneBy, persistToShard, removeFromShard, flushShard complete the surface; each call resolves the shard ID internally and routes the call to the right EntityManager.

Cross-shard reads with ShardedFinder

use Elrise\Bundle\DoctrineShardManager\Finder\ShardedFinder;

final class ReportingService
{
    public function __construct(private ShardedFinder $finder) {}

    public function streamAllActiveUsers(): \Generator
    {
        $shardIds = ['shard_0', 'shard_1', 'shard_2', 'shard_3'];

        return $this->finder->find(
            static fn (Connection $c) => $c->iterateAssociative(
                'SELECT * FROM users WHERE active = 1 ORDER BY id',
            ),
            $shardIds,
        );
    }
}

ShardedFinder is sequential in v1 (per ADR-0004 §Decision). The generator holds one row at a time across the fan-out, so memory is bounded by the largest single-shard result set. Failures throw with the per-shard context framed in the message.

Extending with a custom strategy

services:
    App\Infrastructure\Sharding\TenantPrefixedStrategy:
        tags:
            - { name: app.shard_strategy, alias: tenant }

use Elrise\Bundle\DoctrineShardManager\Strategy\AbstractShardStrategy;

final class TenantPrefixedStrategy extends AbstractShardStrategy
{
    #[\Override]
    public function resolveShardId(mixed $key, array $options = []): ?string
    {
        $tenantId = $this->normalizeKey($key);

        return 'tenant_'.$tenantId;
    }
}

Then reference it as strategy: tenant in sharding_configs.<FQCN> or #[Sharding(strategy: 'tenant', ...)]. The full worked example is in docs/adr/0002-shard-strategy-extensibility.md.

Console commands

The bundle registers two Symfony Console commands on build():

bin/console shard:add <id>

Provisions a new shard's schema. Filters entity metadata to #[Sharding]-annotated classes only and runs SchemaTool::updateSchema($filteredMetadata, saveMode: true).

bin/console shard:add shard_3

Safety warning. shard:add is drop-and-recreate, not additive migration. Run it on greenfield shards only — running it on an existing shard deletes the rows. For an existing shard, use bin/console shard:migrate <id> instead (Wave 2 / ADR-0003). See docs/RUNBOOK.md::Failure Mode 10 for the full migration procedure.

bin/console shard:migrate <id>

Runs Doctrine Migrations on a single named shard. Additive; safe to run against an existing shard.

bin/console shard:migrate shard_3
bin/console shard:migrate shard_3 --dry-run            # print the plan without applying
bin/console shard:migrate shard_3 --migration-set=latest  # explicit version alias

Each command supports the standard --dry-run, --migration-set=<alias-or-version>, and exit-code conventions of the underlying Doctrine Migrations tooling.

Compatibility

• PHP 8.3+
• Symfony 7.2 (7.2.* pin in composer.json)
• Doctrine DBAL 4.2+
• Doctrine ORM 3.3+
• Doctrine Migrations 3.7+
• PSR-6 CacheItemPoolInterface (optional but recommended for hot-path strategies)
• PSR-3 LoggerInterface (optional; graceful degradation when absent)

Known compatibility gaps

• The doctrine/event-subscriber path (modern replacement for the deprecated ShardResolverListener) is planned for Wave 1. Until then, the listener remains @deprecated but is wired in services.yaml for backward compatibility with legacy consumers.
• The three-tier testing infrastructure (unit + bench + itests; ADR-0001) is partially landed. bench/ and itests/ are now populated; the bounded-concurrency CI smoke (DE-011-B) is open and tracked under Issues/open/developer/.
• phpstan and php-cs-fixer are declared in require-dev but the CI-gated enforcement job (Wave 6 / DE-019) is open.

Benchmarks

The bundle ships an in-process benchmark suite under bench/ that exercises the hot path of every public strategy and orchestrator (see ADR-0001). It uses PHPBench and runs without a database by default — subjects that need a Connection opt in via BENCH_DATABASE_URL.

Running

composer bench

The script invokes phpbench run --report=aggregate. The default runner.path (bench/) is configured in phpbench.json, so the positional bench/ argument is no longer required. For custom runs (extra --revs, --iterations, --filter):

php vendor/bin/phpbench run --report=aggregate --revs=3 --iterations=2
# or, to filter a single subject:
php vendor/bin/phpbench run --report=aggregate --filter=ResolveCacheHit

The HTML report and per-subject memory samples are written to bench/.bench/ (gitignored). Subjects honour --filter=<substring> for targeted re-runs.

Latest results (2026-07-21, PHP 8.5.8 NTS, xdebug off, opcache off)

20 subjects, 0 failures, 0 errors. mode is the per-subject median; rstdev is the relative standard deviation across iterations.

Reading the table

• benchResolve* subjects measure strategy hot paths — sub-millisecond per call in a single PHP process.
• benchSwitchToShard* and benchGetConnectionForShard* measure per-shard routing inside ShardConnectionManager; the lookup is amortised to ~1 µs because the manager keeps a WeakMap cache.
• benchWithShard and benchWithEntity measure ShardContext callback wrapping only — no Doctrine round-trip.
• benchFindOneById is the only end-to-end subject that touches a real EntityManager; the value is dominated by Doctrine hydration.
• benchFanOutOver16Shards1kRowsEach exercises ShardedFinder against a stubbed Connection per shard and serves as a regression guard for the fan-out state machine in ADR-0004.

Re-run after any change to src/Strategy/*, src/Resolver/*, src/Manager/*, or src/Context/* and update the table if any subject regresses by more than its published rstdev.

Integration tests

itests/ is the third tier of the testing infrastructure (ADR-0001): end-to-end scenarios that drive the bundle's real classes against real MySQL 8.4 and PostgreSQL 16 instances under controlled concurrency, with a fixed JSON envelope. The scenarios speak to DBAL directly — the bundle ships no Symfony app.

Prerequisites

• Docker Engine with the Compose plugin.
• PHP extensions pdo_mysql and pdo_pgsql enabled (the in-repo composer.json runtime does not load them by default; see Known issues below).

Bootstrap

make itests-up           # docker compose up -d --wait (mysql 8.4 + mariadb 10.11 + postgres 16)
bash itests/bin/migrate-up.sh

Default ports are offset (33061 for MySQL, 33062 for MariaDB, 54321 for PostgreSQL) so the stack coexists with the dbal-manager project's stack on the same host. Credentials default to root:itests for MySQL and MariaDB, itests:itests for PostgreSQL.

Running a scenario

php itests/scenarios/<name>.php \
  --dsn="mysql://root:[email protected]:33061/itests" \
  --vendor=mysql --rows=200 --chunk=50 --warmup --reset

Each scenario prints one JSON envelope line on stdout with scenario, db_vendor, rows, chunk, duration_s, ops_per_sec, peak_rss_bytes, and errors. The full envelope (including DB counters and latency percentiles) is produced by the itests/bin/run-scenario.sh wrapper; see itests/README.md for the CI smoke gate contract.

Latest results (2026-07-22, PHP 8.5.8 NTS, rows=200, chunk=50, warmup+reset)

All 12 scenarios pass on MySQL, MariaDB, and PostgreSQL.

Reading the table

• shard_resolution_* subjects measure the strategy hot path under real DB I/O. MySQL is faster on the hash variant; PostgreSQL is consistently faster on the range and custom strategy variants; MariaDB sits between the two with low variance across all four.
• bulk_write_per_shard is a write-heavy workload that exercises the routing-table fan-out (bench_itests_shard_routing). MariaDB is the fastest at this profile (~2881 ops/s, ~11.5× faster than MySQL on the same workload); PostgreSQL is ~3.3× faster than MySQL.
• cross_shard_lookup validates that the shard ID resolved at write time matches the one resolved at read time across every fan-out shard — see itests/README.md for the cross-shard bleed detection contract.

Known issues

• itests/bin/run-scenario.sh wrapper is unusable as-shipped. The wrapper passes --dsn "<value>" (whitespace-separated) to the underlying PHP CLI, but runner_base.php declares getopt('', ['dsn::', …]) which silently drops the value for long options with optional arguments on PHP 8.5. The wrapper prints "scenario did not emit JSON" (rc=2) and exits. Workaround: invoke the scenario script directly with --dsn=<value> (equals-sign form). Fix: either change 'dsn::' to 'dsn:' in itests/runner/runner_base.php:52 or rework the wrapper to pass DSN via env.
• pdo_mysql / pdo_pgsql are not auto-enabled. The runtime image ships the .so files in /usr/lib/php/modules/ but /etc/php/conf.d/ is empty in the default install. Set PHP_INI_SCAN_DIR=~/.php-confd with one extension=<name>.so line per file, or install the distro packages (php-mysql, php-pgsql).
• composer bench is unusable as-shipped. phpbench.json does not declare runner.path, so composer bench aborts with "You must either specify or configure a path". Pass bench/ explicitly, or add "runner.path": "bench" to phpbench.json.

Important notes

• The sharding_configs array is always required even if you rely solely on #[Sharding] — an empty [] is the canonical declaration. The YAML-vs-attribute precedence test (DE-013, closed) lives in tests/Resolver/ShardResolverTest.php.
• connections and entity_managers are passed as Doctrine service IDs (e.g. @doctrine.orm.default_shard_0_entity_manager), but the per-shard connection and EM MUST also be declared in config/packages/doctrine.yaml so that Doctrine itself instantiates them.
• ShardContext::withShard and withEntity route the active Connection and EntityManager for the duration of the callback only. The previous routing is restored in finally. Cross-shard transactions are out of scope by design (see docs/architecture.md::Non-Goals).
• The deprecated ShardResolverListener class is kept for backward compatibility with consumers wiring doctrine.event_listener. New code should rely on ShardedRepository or ShardContext directly. The listener will be removed in 2.0.

License

MIT — see LICENSE for the full text.