Queue Sql Laravel Package

Technical Evaluation

Architecture fit — The package is a thin async write layer between the query builder and the queue: DB::table(...)->queue(...)->upsert(...) swaps a synchronous write for a chunked, dispatched job. It rides Laravel's existing job/batch/Horizon primitives rather than introducing new infrastructure, daemons, or storage. That is the right shape for our stack — no new operational surface.

The headline is correctness, not CLI — Laravel queues are at-least-once. Prior insert() terminals therefore duplicated rows on any retry, which forced teams into bespoke dedupe or insertOrIgnore workarounds. The upsert() terminal makes each chunk idempotent, converting at-least-once delivery into effectively-once for tables with a natural key. This is the release's real value.

Integration feasibility — Additive API in a minor version; existing insert call sites are untouched. Two hard prerequisites: (a) the target table needs a unique/primary index covering uniqueBy; (b) the driver must support Laravel's upsert — well supported on MySQL/Postgres/SQLite, with known gaps in Laravel's SQL Server grammar.

Technical risk — Moderate-low. Chief risks:

• Writes now depend on unique-index discipline that previously was optional.
• Cross-chunk ordering is undefined if the same key appears in more than one chunk (last-write-wins is not guaranteed).
• update: ['views'] silently narrows what an existing row receives — a schema-drift trap when columns are added later.
• MySQL upsert concurrency: gap locks and deadlocks rise with per-table worker parallelism.
• maxJobs truncation semantics are unverified — silent data loss vs. explicit failure matters a lot here.

Key questions

• Is the --json payload a versioned contract, or incidental output we shouldn't alert on?
• Does upsert fail loudly when no matching unique index exists, or degrade quietly?
• What happens when a key appears twice within a single chunk?
• Is there documented behavior/fallback for unsupported drivers?
• Does maxJobs bound-and-drop, or bound-and-fail?

Integration Approach

Stack fit — Works with database, Redis, and SQS queue backends. Horizon tag parity (queue-sql:upsert:{table}) means existing dashboards and alert rules pick up the new terminal with zero dashboard changes — a meaningful adoption accelerant.

Migration path — No breaking changes, so the package upgrade itself is routine. Adoption of upsert is opt-in per call site, so rollout can proceed table by table rather than as a flag day.

Compatibility — Note that upsert binds more parameters per row than insert. Verify chunk size against SQLite's variable limit and MySQL's max_allowed_packet; a chunk size that was safe for insert may not be safe here.

Sequencing (the package upgrade is not the long pole — the index work is):

1. Upgrade the package; behavior unchanged, zero risk.
2. Wire status --json into existing monitoring and canary-assert the payload shape.
3. Audit candidate tables for a natural key; dedupe existing rows; add unique indexes (online DDL for large MySQL tables — schedule separately).
4. Pilot upsert on one high-retry table, ideally one that has already produced duplicate-row incidents.
5. Roll out per table; retire bespoke dedupe/insertOrIgnore workarounds as each lands.

Operational Impact

Maintenance — Small, effectively single-maintainer package; that is a bus-factor and longevity risk. Pin the version, vendor the source, and be prepared to fork. The additive, well-scoped release history is reassuring so far.

Support — --json is the operational unlock: alert on stuck/failed batches programmatically instead of screen-scraping human output. Treat field names as a contract and add a schema assertion to CI so a future minor release can't silently break alerting.

Scaling — Upsert is heavier per row than insert (index probe plus update path). Guidance: keep chunks ~500–1000, sort rows by key before dispatch to reduce deadlock windows on InnoDB, and cap worker concurrency per target table. maxJobs continues to bound fan-out.

Failure modes — Missing unique index produces a failed chunk job (acceptable if loud). Duplicate key within a chunk yields an indeterminate row and should be prevented upstream. The important improvement: a partial-chunk failure is now retry-safe, eliminating the "half-written table" failure mode that existed with insert. --watch correctly renders once on non-TTY, so the cron/redirect safety hazard is already handled by design.

Ramp-up — Low for engineers (fluent, familiar API). Moderate for on-call: they need a working model of batch naming, chunk-level retries, and reading status --json. One runbook page and a "why is my upsert job retrying?" section covers it.