Screentest Cli Laravel Package

Technical Evaluation

Architecture fit. screentest-cli is a Composer-distributed Laravel Zero binary (screentest) that automates a pipeline a human would otherwise do by hand: scaffold a throwaway Laravel + Filament project from a filakit kit, install the target plugin, run publishes/post-install commands, build assets, seed models, serve, drive headless Chrome (via a Docker-based Puppeteer stub) across each configured URL in light and dark themes, then splice the resulting PNGs into the plugin's README between a marker. Configuration is a single screentest.json at the plugin repo root. Architecturally this is developer tooling, not application code — it never enters a production request path, which caps the blast radius and makes it a low-risk adoption decision for a TPM. The trade-off is that its value is entirely proportional to how many Filament plugin repos you own and how stale their README screenshots are.

Integration feasibility. High for the happy path, moderate for the long tail. It runs from a single command, supports non-interactive init with flags (--name, --package, --screenshots, --no-readme, --deps) so it can be driven from CI without a TTY, and the bundled AGENTS.md is unusually good at documenting the failure taxonomy (missing publish tags, env-gated resources, empty seed tables, signed routes). Those same docs, however, reveal that success depends on per-plugin knowledge: spatie/laravel-package-tools ships migrations as .php.stub and requires an explicit vendor:publish --tag=<short-name>-migrations, and resources gated behind env('FLAG', false) need those flags injected via install.env. Each plugin will need a bespoke, hand-tuned screentest.json the first time.

Technical risk.

• Evidence inconsistency (highest-priority diligence item). The supplied corpus describes three materially different products: the repo description says "screenshot generation for Filament plugins"; description.md describes a generic route-based visual regression / baseline diff tool with pixel thresholds and viewport matrices; assessment-laravel_dev.md describes guest capture, login detection, and version-argument hardening (a visitor-tracking middleware). These cannot all be the same tool at one commit. Before any planning, pin the commit/tag and read the actual source and screentest.json.stub. Plan for the possibility that the README/description are aspirational and the shippable feature is narrower.
• Adoption and bus factor. 2 stars, single maintainer, and a v1.0.41-style release cadence that suggests fast, low-ceremony iteration. Treat this as unsupported open source: no SLA, no deprecation policy, no guarantee of a Filament v6 path.
• External coupling to filakit kits. The kit name (filakitphp/basev5) encodes a Filament major version. Filament majors bring breaking panel/resource APIs, so the tool's usefulness is gated on a third party keeping kits current.
• Polyglot toolchain. PHP + Composer + Node + Puppeteer + Docker + Chrome. That is four runtimes to keep healthy in CI, and Docker availability inside the runner is now a hard dependency.
• Heuristic discovery. init --deps= walks composer.json requires up to three levels and infers publish tags and env flags. Inference failures are silent by default; the newer 4xx/5xx and Whoops/Ignition detection is a mitigation, not a fix, and it warns rather than fails.
• README mutation. The tool writes into a tracked repo file. That creates review burden, merge-conflict surface, and a temptation to auto-commit binary PNGs.

Key questions.

1. What is the actual scope at the pinned release — README screenshot generation, or baseline visual regression? Both?
2. License and supported PHP/Laravel/Filament version matrix?
3. Can it target an existing application, or only a throwaway scaffold? (The description.md implies the former; AGENTS.md implies the latter.)
4. Chrome execution: Docker-only, or can a host/CI-provided browser be substituted?
5. Is the seed step's admin user ([email protected] / password) guaranteed ephemeral, and does the temp project's .env ever land in an artifact upload path?
6. Is capture deterministic across machines (font rendering, prefers-color-scheme for dark mode, device pixel ratio, locale/timezone)?
7. Does README updating support --no-readme cleanly in CI, and is the marker splice idempotent?
8. What is the runtime cost per plugin (full composer install + npm build + serve + N×themes)?
9. For signed routes, the tool drops a temporary Artisan command into the scaffold and runs screentest:resolve-urls — is that acceptable in your CI sandbox?

Integration Approach

Stack fit. Good if the org standardizes on Laravel + Filament and already ships plugin packages with README screenshot sections. Poor fit if plugins are few, screenshots change rarely (manual capture is cheaper than a CI pipeline), or if Filament versions are heterogeneous across repos — each kit pin multiplies the maintenance matrix. It is a dev/CI-only dependency: add it under require-dev or invoke it via a pinned composer global/vendor/bin path, never in production require.

Migration path. Treat this as additive tooling with no migration of existing code. The only "migration" is social: moving screenshot upkeep from ad-hoc human capture to a generated, reviewable artifact. Preserve the current README screenshot section first (create the <!-- SCREENSHOTS --> marker and commit the hand-made images inside it) so a failed tool run has something to fall back to and the first generated diff is a clean comparison.

Compatibility. Pin the tool to an exact version (no ^), pin the filakit kit name in screentest.json, and record the tool version alongside the Filament major in your repo docs. Because the kit name encodes a Filament major, upgrading Filament across a plugin fleet means re-validating screentest.json per plugin, not just bumping a lockfile. Verify Node/Chrome availability inside your CI image before committing to an approach; if you run on ephemeral GitHub Actions runners, Docker is present, but self-hosted or locked-down runners may not be.

Sequencing.

1. Spike (1–2 days). Clone, read source at the pinned tag, reconcile the documentation inconsistency above, run screentest init --path=. --name=... --package=... --screenshots=all --no-readme against one real plugin end to end. Time it and record every manual fix required.
2. Characterize the config surface. For that pilot plugin, document the confirmed publish tag(s) and any install.env flags. This becomes the template for step 3.
3. Standardize. Publish one canonical screentest.json template plus a short internal runbook (what to check when a screenshot is blank, 404, or 500). Explicitly document the .php.stub publish-tag trap and the "resource gated by env(..., false)" trap — these are the two failures that cost the most engineering time.
4. Pilot on 2–3 plugins, one simple (list page) and one with relations or signed routes, to exercise the seeding and route-resolution paths.
5. Wire CI second, not first. Get humans comfortable running screentest run locally before automating it. When automating, make README updates a PR (bot branch) rather than a direct commit to main.
6. Decide on fork-vs-depend. If the pilot succeeds, budget for a vendored fork with a pinned upstream baseline; a 2-star, single-maintainer tool is a fork candidate the moment it becomes load-bearing across the fleet.
7. Roll out repo-by-repo, adding a checklist item at plugin-release time rather than a hard gate until the pipeline is proven stable for a full release cycle.

Operational Impact

Maintenance. Two recurring costs. First, dependency drift: Composer/Filament major bumps, kit renames, and Puppeteer/Chrome image updates will each need a re-run of the pipeline and probably a config touch. Second, config creep: every plugin's screentest.json is hand-maintained and encodes publish tags and env flags that mirror that plugin's internals — when a plugin adds a resource or a feature flag, the config silently under-captures until someone notices a missing screenshot. Budget a recurring "regenerate and eyeball" task per release, not a one-time setup.

Support. No commercial support, no SLA, 2 stars. Assume no upstream responsiveness for bugs and no roadmap for Filament v6. Mitigation: pin versions, vendor a fork, and keep the internal runbook as the primary support artifact rather than upstream docs. The AGENTS.md file is genuinely useful here — it is written for AI coding agents but doubles as a troubleshooting checklist for humans, which materially lowers onboarding cost.

Scaling. Cost is roughly linear in plugins × screenshots × themes, and each run pays a full scaffold cost (Composer install, asset build, migrations, seeding) before a single page is captured. That is the dominant expense — minutes per run, not seconds. Optimize in this order: (a) restrict screenshots to the pages that actually matter rather than all, (b) cache Composer and npm layers in CI, (c) prefer a single-theme run for routine PRs with the light+dark matrix reserved for release tags, (d) run on a schedule (nightly/weekly) rather than per-push if drift is slow. Beyond a dozen repos, expect a real CI-minutes line item and consider centralizing runs in one scheduled job that fans out.

Failure modes (ranked by expected pain, with the signal you get):

• Blank or identical error screenshots from an unpublished migrations tag — every listing page renders the same generic error. Mitigated by the 4xx/5xx and Ignition-page warning, but that only warns; the PNGs are still written. Make the warning fail the job in your CI wrapper.
• Single-resource 404 from a resource gated behind an env flag defaulting to false. Scope is one screenshot, and it is a true 404 — a good tell to teach in the runbook.
• /1/edit-style URLs with empty tables because seeding partially failed (required relations the factory does not fill). Check row counts, don't trust the "Generating and running seeds ✔" line.
• Silent staleness — the pipeline succeeds but the screenshot is months out of date because the URL was never updated when routes moved. There is no diffing baseline in the documented (plugin-screenshot) mode; a reviewer is the only control.
• Non-deterministic capture — font rendering, dark-mode prefers-color-scheme emulation, and viewport differences produce churn in a binary artifact. If you adopt binary PNGs in git, evaluate LFS before history bloat becomes a problem.
• README churn — regenerated images create noisy PRs and merge conflicts on a file humans also edit. Isolate into a bot PR.
• Ephemeral-environment leakage — seeded credentials and .env values live in a temp project; confirm the temp directory is never uploaded as a CI artifact or committed by a broad git add.

Ramp-up. For an engineer who has shipped a Filament plugin: half a day to first successful capture, one to two days to a reliable config including publish tags and env flags. For someone unfamiliar with Filament panel registration and spatie/laravel-package-tools publish semantics, expect two to three days, mostly spent on the two silent-failure traps. The TPM-side ramp-up is smaller but non-trivial: the decision-maker needs to internalize that this tool produces documentation artifacts, not test coverage, and that the corpus's claim of baseline pixel-diffing should not be budgeted for until verified against the pinned source.