Process Records Laravel Package

Getting Started

Install (private Bitbucket repo — you need credentials before Composer can resolve it):

composer config repositories.process-records vcs https://bitbucket.org/bmooreitul/process-records
composer require itul/process-records

In CI, export COMPOSER_AUTH with a Bitbucket OAuth consumer or app password; anonymous clones of bmooreitul/* will 404 and look like a missing package.

Publish and migrate — almost every package of this shape ships both:

php artisan vendor:publish --provider="Itul\ProcessRecords\ProcessRecordsServiceProvider"
php artisan migrate

Where to look first, in this order:

1. config/process-records.php — table name, connection, polling interval, route prefix/middleware, and whether progress is persisted to the DB or to the cache store. This file answers 80% of the questions you'll have later.
2. The migrations — the column set tells you what the package can actually express (status enum? progress as int percentage or current/total counters? message? payload JSON?).
3. src/Models/ProcessRecord.php (or equivalent) — the public API you'll be calling from jobs: create/start, update progress, complete, fail.
4. The routes file — the endpoint your JavaScript will poll, and whether it's already behind auth/web.

First use case — the canonical one, a chunked import:

use Itul\ProcessRecords\Models\ProcessRecord;

$record = ProcessRecord::create([
    'name'    => 'Customer import',
    'status'  => 'running',
    'total'   => $rows->count(),
    'current' => 0,
]);

foreach ($rows->chunk(500) as $chunk) {
    // ... work ...
    $record->increment('current', $chunk->count());
}

$record->update(['status' => 'completed']);

Then hit the package's status route from the browser and render the returned percentage. Confirm the exact method names against the README — this package has no public docs beyond the repo, so treat the source as the spec.

Implementation Patterns

Dispatch-and-poll (the pattern this package exists for). Never run the long job inside the request that renders the page. Create the record in the controller, hand its ID to a queued job, return the ID to the view, and let the browser poll:

public function store(ImportRequest $request)
{
    $record = ProcessRecord::create([
        'name'   => 'Import: '.$request->file('csv')->getClientOriginalName(),
        'status' => 'queued',
        'total'  => 0,
    ]);

    ImportCustomers::dispatch($request->file('csv')->path(), $record->id);

    return response()->json(['id' => $record->id]);
}

const timer = setInterval(async () => {
    const r = await fetch(`/process-records/${id}`, { headers: { Accept: 'application/json' } });
    const p = await r.json();
    bar.style.width = `${p.percent ?? 0}%`;
    if (['completed', 'failed'].includes(p.status)) {
        clearInterval(timer);
        if (p.status === 'failed') showError(p.message);
    }
}, 1000);

Always clear the interval on terminal status and on visibilitychange/page unload — an orphaned poller is the most common source of noise in the logs.

Throttle writes inside the loop. Updating the row on every iteration turns a 50k-row import into 50k UPDATEs. Update only when the percentage actually changes, or every N items:

$lastPct = -1;
foreach ($rows as $i => $row) {
    process($row);
    $pct = (int) floor(($i + 1) / $total * 100);
    if ($pct !== $lastPct) {
        $record->update(['current' => $i + 1]);
        $lastPct = $pct;
    }
}

Use it from Artisan too. For php artisan import:customers the same record doubles as a live status page your ops team can watch, and as an audit trail of past runs. Create the record at the top of handle(), mark it failed in catch, and it becomes a poor man's job history.

Wrap the whole job body so failure is always recorded:

try {
    // work
    $record->update(['status' => 'completed', 'finished_at' => now()]);
} catch (\Throwable $e) {
    $record->update(['status' => 'failed', 'message' => Str::limit($e->getMessage(), 500)]);
    throw $e;
}

Also hang the same logic off failed() on the job, because a fatal (OOM, timeout) never reaches your catch.

Multi-step workflows. For pipelines, create one parent record per stage and aggregate for the UI, or a single record whose message you mutate ("Extracting…", "Validating…") and whose total you reset per stage. Aggregating child rows is cleaner if the schema has a parent_id; otherwise compute progress as completed_stages / total_stages in your own view model.

Front-end integration. If the package ships a Blade partial, prefer including it over hand-rolling markup — it will pick up the polling URL and CSRF handling correctly. If you need custom UI, keep the package's route as the data source and style your own component; don't fork the partial, because updates will fight you.

Pair it with Bus::batch() when fanning out. If work is split across many jobs, use the batch for orchestration and the process record purely as a display/telemetry row updated from then/catch callbacks. Don't let two sources of truth (batch counters and record counters) drift — pick one to write and let the other read it.

Gotchas and Tips

Stuck-in-running records are the #1 support ticket. Anything that kills the PHP process — OOM, max_execution_time, a queue worker restart, a deploy — leaves the row at running forever, and the browser polls indefinitely. Mitigations: (a) always set a failed() handler; (b) add a last_heartbeat_at or lean on updated_at and treat rows older than N minutes as stale both in the poller response and in the UI; (c) ship a scheduled cleanup command that fails records whose updated_at is older than the longest plausible run.

Queue timeouts must exceed job runtime. --timeout on the worker, retry_after in config/queue.php, and max_execution_time/fastcgi_read_timeout all need headroom. A job killed at 60s mid-import looks like a package bug but is a worker config bug.

If you run it synchronously in a web request, release the session. PHP's session lock means a second request from the same tab blocks until the first finishes — so your poller hangs and the progress bar never moves. Call session()->save() (or use a route without session middleware) before the long loop. This is the single most confusing failure mode for first-time users.

Polling endpoint auth is on you. If the package registers its routes without middleware, process names and payloads can leak to anyone who can guess an ID. Add auth in the config's middleware key, and don't expose auto-increment IDs — prefer a UUID/ULID if the schema offers one.

Multi-server deployments. DB-backed progress works fine across servers; anything cached in-memory or static per-process does not. If the config lets you choose a store, pick the DB or Redis, not array.

Private package + CI. Fresh clones and CI runners need COMPOSER_AUTH; also remember composer.lock will reference a Bitbucket commit hash, so tagging matters for reproducible deploys.

Config quirks. Publish config before migrating, and re-run php artisan config:cache after editing it — stale cached config silently points at the wrong table. Check for table-name collisions if you already have a processes or jobs table.

Don't poll faster than you write. If the record updates once per second, polling at 200ms just triples query load for no visible gain. 1–2s is right. Consider switching to broadcasting (Laravel Echo) if the package exposes events on status change — the poller is fine for admin tooling, wasteful for anything high-traffic.

Testing. Use RefreshDatabase, freeze time where the package uses updated_at for staleness, and assert on the terminal status rather than on intermediate progress values, which are inherently racy. For the polling endpoint, hit it directly with actingAs() and assert the JSON shape.

Extension points to use before forking. Typical knobs: binding a subclass of the model in the container, overriding the published Blade view, swapping the route middleware, and configuring the table/connection. If progress semantics don't match your needs (e.g. you need weighted steps rather than item counts), write a thin domain wrapper around the record rather than patching vendor code — with a zero-star, single-maintainer package, forks never receive upstream fixes.