allnetru/laravel-sharding
A Laravel package for horizontal database sharding. Route queries and Eloquent models across multiple database connections by shard key, manage sharded migrations, and scale large datasets while keeping a familiar Laravel workflow.
Architecture fit. The v0.6.4 note confirms this is a shard-routing package: it intercepts Eloquent/query-builder calls, inspects the query shape, decides whether a query can be answered from a single shard or must fan out, and rewrites or refuses aggregates accordingly. That places it in the data-access layer beneath your domain code, which is the right altitude for a Laravel application — it means feature teams keep writing ordinary Eloquent and inherit shard-awareness rather than adopting a bespoke repository abstraction. The package exposes meaningful configuration surface (pin_by_key) and lifecycle operations (rebalancing), which implies it is designed as infrastructure, not a library.
Integration feasibility. Reasonable for a greenfield or already-partitioned domain. The failure mode it fixes is precisely the one that bites real teams: a query that is logically single-shard — where('user_id', 5)->distinct()->count('letter_id') — was refused because the guard reasoned about query shape (presence of distinct/group by/having) rather than about actual shard fan-out. That class of bug produces a hard exception in application code, discoverable only at runtime, in exactly the reporting endpoints product wants most (per-tenant distinct counts, grouped rollups). That it reached a patch release suggests the test matrix is being driven by real usage rather than by a formal routing model.
Technical risk. Three items, in order:
avg() is now "taken from its shard whole rather than as a sum and a count from two queries, which on a grouped query need not be the same group." That is the correct behaviour, but it means any dashboard or report built on the old decomposition would have produced subtly wrong numbers rather than an error. Treat every aggregate on a sharded model as needing an explicit correctness review — wrong numbers are worse than exceptions.pin_by_key is off and the query fans out again. Product behaviour therefore depends on cluster state at request time. Any user-facing feature that depends on a keyed aggregate needs a defined degraded mode.Key questions for the maintainer / vendor.
readsOneShard()) the single authority for both raw-aggregate and grouped-aggregate refusals, and is there a documented truth table of query shape × key presence × cluster state → routed/refused?RuntimeException?pin_by_key have a safe default, and what happens to in-flight queries when it is toggled?Stack fit. Assumes a Laravel application with Eloquent as the system of record and a MySQL/Postgres backend capable of multiple databases or schemas. The package is transparent to controllers, jobs, and policies — the work concentrates in models and in the configuration file. It is a poor fit if the team is heavily invested in raw query-builder strings or DB-facade calls that bypass the model layer, because those bypass the router too.
Migration path. Do not introduce this as part of a feature release. Sequence it as its own workstream:
DB::raw / DB::select aggregates on sharded tables without an explicit opt-out comment.Compatibility. Verify against the Laravel and PHP versions in production before the spike — 0.x packages commonly lag major framework releases. Confirm the database driver matrix; sharding packages often support the primary driver fully and others partially.
Sequencing against product. Ship the package behind routing for new features first, where there is no legacy query corpus to audit. Backfill existing features only after the refusal inventory is empty or each item has an accepted workaround.
Maintenance. This is a dependency that sits on the critical path of every database call, so it warrants the same change-control as the ORM itself: pin the version, read every changelog, and never auto-merge a 0.x minor. Assign a named owner. Realistically one engineer at 10–20% time for stewardship plus upgrade testing.
Support. The failure signatures are distinctive and should be documented in the runbook before launch: the «cannot be combined across shards» exception, and its appearance during a rebalance as a cluster-state symptom rather than a code defect. Without that, an on-call engineer will spend the first hour of an incident in the application code instead of the shard topology. Ask the vendor for a named escalation contact; a 0.x package may be one maintainer.
Scaling. Shard routing scales reads and writes roughly linearly for keyed access, and not at all for cross-shard aggregates. Any product requirement that needs a global count, global average, or cross-tenant leaderboard is outside this package's competence and needs a separate mechanism — a rollup table, a warehouse, or a precomputed counter. Establish that boundary with product early; it is an architectural constraint, not a backlog item.
Failure modes.
avg() case). Detect with a one-off reconciliation job comparing sharded results to a single-DB copy on a sample.pin_by_key off silently converts single-shard reads into fan-outs and reintroduces refusals. Protect this flag with a config test.Ramp-up. Two to three days for a competent Laravel engineer to be productive: shard-key design, the routing rules, and the aggregate constraints. Add a half-day team session on "why your distinct()->count() sometimes throws," because that is the piece engineers will hit first and understand last. Total elapsed time from approval to production, including the spike and rebalance rehearsal: six to ten weeks for a mid-sized application.
How can I help you explore Laravel packages today?