Operations Laravel Package

Getting Started

Get the tracking table in place before anything else — the whole run-once guarantee hinges on it.

composer require directorytree/operations
php artisan vendor:publish --tag=operations-migrations   # confirm the tag against the source
php artisan migrate

Then confirm the command surface for your installed version before writing code:

php artisan list operations        # exact signatures for runner + status
php artisan list make              # confirm make:operation exists and its flags

Scaffold and run:

php artisan make:operation BackfillUserTimezones
php artisan operations:run
php artisan operations:status

A generated operation lands in app/Operations and is a plain class with a handle() method:

namespace App\Operations;

use DirectoryTree\Operations\Operation;

class BackfillUserTimezones extends Operation
{
    public function handle(): void
    {
        User::query()
            ->whereNull('timezone')
            ->chunkById(500, fn ($users) => $users->each->update(['timezone' => 'UTC']));
    }
}

Where to look first: (1) the published migration — read the table schema, it tells you what the runner actually records, whether there's a unique constraint, and (as of v1.2.0) where checkpoint data lives; (2) the base Operation class — lifecycle hooks, whether a transaction is opened around handle(), what dependencies are injected, and the checkpoint API surface (checkpoint(), resume(), or whatever the actual methods are named — confirm against source); (3) the runner command class — this answers the concurrency question (lock? atomic insert? best-effort?) faster than any README will. With 37 stars and a single maintainer, the source is the documentation. Read it once, thoroughly, before it goes anywhere near production.

First real use case: a backfill or data correction you'd otherwise paste into php artisan tinker and then comment out a Slack message saying you did it. Something too stateful for a migration and too one-off for a seeder.

Implementation Patterns

Hold the schema/data line permanently. Migrations run on every fresh environment and describe structure; operations run once per environment and describe state transitions. Backfills, re-encrypting a column after key rotation, remapping third-party IDs, warming caches, normalizing legacy rows, deleting data orphaned by a removed feature flag — all operations. If it must be true on every environment including a fresh clone, it's a migration or seeder, not an operation.

One operation, one intent, one PR. Small classes are reviewable. Name them imperatively with the why baked in: BackfillUserTimezonesAfterUtcCutover beats FixData2. Keep a short README.md next to app/Operations with a one-line index: class, date merged, environment(s) applied, rollback note. By operation #30 that index is the only thing standing between you and unreadable tribal history.

Keep operations short; push heavy work to jobs. A twenty-minute backfill inside a deploy hook blocks the release and invites pipeline timeouts. Pattern instead: the operation's handle() dispatches a queued job (or a batch) and returns. The operation records "the decision to run this work was made"; the queue does the work with retries and its own observability. Checkpoints do not change this advice — they make a long operation recoverable, not fast, and an inline long operation still blocks your deploy.

Chunk by primary key, never by offset. chunkById(500, ...) is the default shape for anything touching a table you can't lock. Avoid ::all() and unbounded update() statements — on a large table the latter will hold locks that turn a routine deploy into an incident.

Use checkpoints for anything that runs more than a few minutes (v1.2.0+). Opt-in checkpoints persist progress and let a failed operation resume from where it stopped on the next run instead of re-processing from the top. The shape:

public function handle(): void
{
    $cursor = $this->checkpoint('cursor', fn () => User::min('id'));

    User::query()
        ->whereNull('timezone')
        ->where('id', '>', $cursor)
        ->chunkById(500, function ($users) use (&$cursor) {
            $users->each->update(['timezone' => 'UTC']);
            $cursor = $users->last()->id;
            $this->saveCheckpoint('cursor', $cursor);
        });
}

Treat the checkpoint as the last fully-committed unit of work, and advance it only after that unit's side effects are durable. Confirm the exact persistence semantics in the source (per-chunk write? per-operation snapshot? what column/table?) before designing around it — the release note describes capability, not mechanics. Checkpoints are the answer to the partial-application failure mode, but only if your chunk boundaries and your checkpoint writes line up. Off by one chunk, and a resume silently skips rows.

Write each operation to be safe to run twice. The run-once guarantee is a promise from the package; your resilience against a restored database, a concurrent pod, or someone running it manually comes from the operation itself. whereNull('timezone'), updateOrCreate, upserts with unique keys — cheap insurance, and it makes the operation testable in isolation. Checkpoints reduce the need to re-run from the top, but they don't remove the need to be idempotent at the chunk level — the operation can still fail mid-chunk after partial writes.

Test operations like any other application class. They're plain classes with resolvable dependencies, so (new BackfillUserTimezones)->handle() under RefreshDatabase plus fixtures is enough. Assert the post-state, not the SQL. For checkpointed operations, add a second test: seed the checkpoint, run, assert that already-processed rows are untouched and remaining rows are. This is the single highest-value habit here: operations live outside your request-path coverage, so without tests a regression surfaces weeks later as a data anomaly nobody can trace.

Put the runner after migrate in the deploy pipeline, gated for production:

php artisan migrate --force
php artisan operations:run --force    # confirm flag support in your version

Resolve collaborators through the container. If an operation needs an API client or a service, type-hint it rather than app()-ing it mid-method — that's the "resolves dependencies through Laravel's container" claim actually paying rent, and it makes the test double trivial.

Gotchas and Tips

The concurrency question is the one that bites. Whether the runner serializes execution with a lock or an atomic insert determines whether a rolling multi-pod deploy double-executes an operation. Read the runner source and answer it explicitly. If the answer is "best effort," wrap the deploy step in your own guard: run operations in a single-job pipeline stage, or bracket the call in Cache::lock(...). Don't leave this to hope. Checkpoints make this worse if you get it wrong: if two pods share a checkpoint, one can resume from a cursor the other just advanced past — rows get skipped, not just double-processed. Concurrency safety is a prerequisite for trusting checkpoints, not an alternative to it.

Transaction semantics are worth knowing before your first DDL-ish operation. If the package wraps handle() in a transaction by default, be aware MySQL implicitly commits on DDL and Postgres will refuse certain statements — and long transactions over big backfills will bloat. Check the source; opt out where it doesn't fit. If checkpoints are stored in the same DB and the operation is transacted, confirm whether the checkpoint write joins or escapes the transaction — a checkpoint committed inside a rolled-back chunk is a resume that skips work.

Checkpoint the right thing, and only for the right operations. Checkpoints are opt-in for a reason: they add a schema surface, they accumulate rows, and they lie to you if the underlying data changed between runs (a chunk of rows edited between the failure and the resume). Use them for long, monotonic, cursor-ordered backfills. Don't add them to a two-second data fix — the bookkeeping outlives the value. When an operation is done for good, prune its checkpoint state; an orphaned checkpoint is a future reader's confusion.

Environment drift is silent. Records live in the database. Restore prod into staging and staging now believes prod's operations already ran against staging's data — they didn't. Checkpoints travel in the same restore, so a restored environment can resume an operation against data that has nothing to do with the checkpoint's cursor. Any time you clone a database, either truncate the operations (and checkpoint) tables or document loudly that the state is inherited. Same story for multi-environment promotion: staging and production each keep their own ledger, which is a feature, but only if you remember it.

Forward-only, by design. There is no down(). Rolling back code does not roll back an operation — and does not roll back a checkpoint either. Every operation PR should carry a written rollback note in the description — the inverse SQL, or an explicit "irreversible, here's why we're confident." The on-call engineer needs it at 2am, not you.

Decide fail-closed vs. skippable before the first failure. A failing operation halts the deploy. For a runbook-critical backfill, failing closed is right. For a cosmetic cache warm, it's not. Encode the choice rather than discovering it mid-release. Resume-on-next-run is a response to failure, not a replacement for deciding how the pipeline behaves when the first run fails.

Partial application is the ugly failure mode — checkpoints mitigate it, they don't eliminate it. A non-transactional operation that dies halfway leaves data the ledger doesn't describe — the table says "not run," the data says "half run." Checkpoints close most of that gap for the happy resume path: a chunked, checkpointed operation restarts from the last committed cursor. But the operations that can't be cleanly checkpointed — ones that touch multiple tables, call external APIs, or fan out — still need the old discipline: chunked, idempotent, resumable by construction. And the checkpoint itself is a claim about reality; verify it before trusting it. Long operations should log progress (see below) regardless.

Log inside handle(), and log checkpoint advances. This code runs once, at deploy time, often at night, with nobody watching stdout. Structured log lines at start/finish, per chunk, and — critically — at every checkpoint save (from cursor, to cursor, chunk size, rows touched) are your only forensics. On a resume, that log is the only reliable way to know what actually got processed versus what the checkpoint claims. Capture output in the pipeline too.

Pin the version and stay fork-ready, but read release notes. Zero dependents, one maintainer, a portfolio of other packages competing for attention. Use an exact constraint, not ^1.0, so an upstream release can't quietly change deploy behavior on a Friday. v1.2.0 is a good example of why: checkpoints are opt-in and additive, so it's low-risk, but a future release could flip a default or change checkpoint storage. Read the release notes on upgrade, not after. The surface area is a provider, a command, a table, a base class, and (now) a checkpoint mechanism — still vendorable in an afternoon if the project stalls. Keep it off your Laravel-upgrade critical path and add a quarterly check that it still supports your major.

Don't let it replace migrations or seeders, and don't let the directory become a dumping ground. Assign one owner. One-off cleanup that could have been a migration becomes, over time, a second and undocumented source of deployment state — and with checkpoints in play, a third undocumented store of progress state that nobody owns.