Queue Sql Laravel Package

Product Decisions This Supports

• Build vs. buy for bulk-write plumbing. v1.4.0's upsert() terminal makes the package a credible answer to "how do we get 100k+ rows into the database without blocking a request or writing our own chunker, retry logic, and batch tracker?" That is a build-vs-buy decision most teams under-cost: the naive version is a week of work plus a permanent tail of edge cases.
• Idempotency as a product guarantee, not an implementation detail. Retry-safe upserts keyed on a uniqueBy column set let you promise "a retried or duplicated job cannot create duplicate rows." That directly supports features where duplicates are user-visible — view counters, daily rollups, usage metering, leaderboard stats — and lets you ship at-least-once queue semantics without a bespoke dedupe layer.
• Roadmap sequencing: defer the data pipeline. If the near-term need is "aggregate app data into summary tables nightly," this package plus Laravel's scheduler buys you quarters before you need a warehouse/ELT investment. That's a real roadmap trade, and this release strengthens the "defer" option.
• Observability investment decisions. queue-sql:status --json turns batch progress into a machine-readable feed, so the question shifts from "do we build custom progress tracking?" to "do we point Grafana/Datadog at this endpoint?" --watch is dev-ergonomics, not production surface (it deliberately degrades to a single render when piped), so don't plan monitoring around it.
• Concrete use cases to scope against: webhook/API ingestion bursts, CSV and backfill imports, denormalized stat tables, per-tenant usage rollups, and syncing external systems into local tables where the source can re-deliver the same records.

When to Consider This Package

Adopt when most of these hold:

• You are on Laravel with a working queue and (ideally) Horizon already in production — the package's value compounds on infrastructure you already operate.
• Writes are bulk and non-interactive: they can tolerate queue latency and you'd otherwise be doing insert loops inside a request or a controller-less artisan command.
• You need chunking plus safe retries: your data has (or can be given) a natural unique key so upsert(uniqueBy: ...) is meaningful.
• Volumes are in the thousands-to-low-millions-of-rows-per-job range where a chunked INSERT/UPSERT over the existing connection is genuinely cheapest.
• You are a small team deliberately avoiding new operational surface — no Kafka, no Airflow, no warehouse.

Look elsewhere when:

• The destination is a warehouse or you need cross-source transforms — use an ELT tool; a PHP job runner is the wrong layer.
• You need true exactly-once semantics, CDC, or ordering guarantees across jobs. upsert gives you idempotent effects on a unique key, which is not the same thing; be precise with stakeholders about the difference.
• Volumes are very large or the load is continuous — native COPY/LOAD DATA, or a dedicated loader, will dominate.
• You have no queue worker discipline (no Horizon, no failed-job triage, no alerting on queue depth). This package moves work into a place you must already be watching.
• Audit, lineage, or regulatory retention requirements apply to the ingested data — you'll want an explicit pipeline with recorded provenance.

Dependency risk to price in: this is a single-maintainer package. Mitigate by pinning the version, confining usage behind a thin internal interface (one class that owns the queue()->upsert()->dispatch() call), and keeping a fallback path to plain chunked upsert in a job. That containment cost is small and should be a stated condition of adoption, not a surprise later.

How to Pitch It (Stakeholders)

To executives: We can keep large data writes off the request path and out of our engineers' backlog without buying or building new infrastructure. The v1.4.0 release closes the failure mode that mattered most — a retried job duplicating rows — so the numbers we show in dashboards and usage reports stay correct even when the queue misbehaves. It also emits status as JSON, which means existing monitoring can watch these jobs without a custom integration. Cost is one small dependency on infrastructure we already run; the main risk is that it's maintained by a single author, which we contain by pinning the version and wrapping it behind our own interface.

To engineering: v1.4.0 adds an upsert() terminal that fans out over rows like insert, but each chunk is an idempotent upsert keyed on uniqueBy — retries no longer create duplicates. It respects chunk and maxJobs, and it propagates queue-sql:upsert:{table} batch names and Horizon tags so batches stay traceable in the tooling we already use. queue-sql:status --json gives us a machine-readable feed for dashboards; --watch is for local debugging and intentionally renders once when output isn't a TTY, so it isn't a production monitoring story. No breaking changes noted in this release. Recommended adoption pattern: one internal wrapper class owns every call site, so an upstream abandonment or a schema change costs us a single file to unwind.