kylekatarnls/multi-tester
multi-tester runs the unit tests of dependent projects against your local package changes by swapping your code into their vendor directory, with Travis CI support.
Architecture fit. Despite the framing, this is not a Laravel package — it is a framework-agnostic Composer dev-dependency and CLI (vendor/bin/multi-tester) that orchestrates other repositories' test suites. The architecture is a thin shell-out harness: read .multi-tester.yml, resolve each target package's Git URL (explicit clone, or Packagist API lookup with a semver version filter), clone into a working directory, composer install in that clone, overwrite the target's vendor/<your-package> with a symlink/copy of your working tree, then run that project's script. There is no plugin API, no daemon, no service — it is a config file plus three commands per project. That is the right shape for a TPM's purposes: it is a pipeline convention, not a platform, and it can be discarded without unwinding an integration. It also means it can drive non-PHP projects (the script is arbitrary), though every ergonomic affordance — replace handling, autoload regeneration, allow-plugins shortcuts — assumes Composer.
Integration feasibility. High for a first spike, moderate for durable CI. --add=<vendor/package> scaffolds config; defaults cover the common case (composer install --no-interaction, vendor/bin/phpunit --no-coverage); install: travis / script: travis / the shorthand travis harvest commands from a downstream .travis.yml, which is useful for OSS consumers you don't control. Multiple versions of the same package are expressible inline (symfony/symfony:5.4.*: default), which is precisely the support-matrix lever a TPM cares about. The blockers are environmental, not conceptual: CI needs outbound access to Packagist/GitHub, a git binary, PHP ≥ 7.2 (7.1 was dropped in 2.6.0), and a composer.json with a name at the configured config.directory. Expect the honest spike cost to be half a day to prove one downstream project, then one to two days to harden it into a required check.
Technical risk. Several items are yellow-to-red and should be written down before adoption:
vendor/ interacts badly with Composer 2 plugin allow-listing, path repositories, replace keys (partially improved in 2.6.1), and autoloader state. This is where silent false-greens live.clone/install/script shell strings in your CI runner. That is by design, but it means config changes are privilege-equivalent to CI config changes and need the same review.version on every entry and treat unpinned entries as a defect.Key questions to resolve before committing.
version and a success_only fallback, or accept HEAD-drift flake?.multi-tester.yml and reviews changes to it, given it contains executable shell?Stack fit. Good fit for a Composer-published library that is a dependency of other teams' applications or packages — the canonical case being a shared internal package consumed by several Laravel services, or a package whose own tests pass while a consumer's service provider, config default, or container binding silently changes behavior. Poor fit for an application repo with no downstream consumers, and poor fit as a substitute for contract or E2E testing. Confirm the "Laravel/PHP package" label maps to a Composer library, not an app — if it is an app, this tool has no target.
Migration path. Non-invasive and reversible. Install as --dev, generate config with --add, commit .multi-tester.yml, add one conditional job to existing CI. Nothing in the shipped artifact or runtime path changes; there is no production dependency and no vendor code shipped to users. Removal is composer remove --dev plus deleting the job — a clean exit, which is the main argument for trying it over building a harness.
Compatibility. Verify three things concretely: PHP version against 2.6.x's ≥ 7.2 floor; the Symfony component constraints the tool pulls in as a dev-dependency (they track Symfony 6/7/8, and a dev-dep conflict with your app's pinned Symfony versions is a real risk — check composer why-not); and Composer 2 behaviour on allow-plugins, which 2.6.1 explicitly patched. For each downstream target, confirm its install/script is genuinely non-interactive and that its own composer install succeeds from a clean clone.
Sequencing. (1) Pick one high-value downstream project with a suite that passes standalone. (2) Run the CLI by hand against it; confirm the vendor swap actually takes effect by introducing a deliberate breaking change and observing a red run — this is the single most important validation step, and skipping it is how teams ship a no-op gate. (3) Pin version and enable stop_on_failure (fail fast) and --quiet-install for log hygiene. (4) Add the CI job as advisory for two weeks. (5) Expand to the full named matrix, one project per PR. (6) Promote to required. (7) Document the fork-and-own contingency.
Maintenance. Low code footprint, but real stewardship cost. The tool itself needs little upkeep; the configuration rots — downstream repos move branches, change test commands, add CI requirements. Assign a named owner and review .multi-tester.yml on a schedule, not just when it breaks. Assume you may need to fork; at this size that is days, not weeks, and MIT licensing removes the legal friction.
Support. Internal support is you, since there is no community to escalate to. The realistic burden is onboarding: a new maintainer needs to understand the vendor-swap mechanism and its false-green modes. Publish a one-page runbook covering how to read verbose output (-v), how to reproduce a target locally, and the known failure signatures: install failing on a private dependency, Packagist 404/rate-limit, and "swap succeeded but tests ran against the old code."
Scaling. Linear and unbounded in the wrong dimension. Each target is a clone plus a full composer install plus a suite run — five targets is a long job; a wide support matrix multiplies it. The README's own pattern is a parallel CI matrix entry dedicated to MULTITEST=on, which is the right shape: isolate it so it does not gate every PR. stop_on_failure and success_only are the two levers for duration and flake; use both. Beyond roughly five to eight targets, consider splitting across CI runners rather than serializing.
Failure modes to instrument. (a) False green — the vendor override silently failed and the target tested its published version; assert the swap by checking that a deliberately broken commit produces a red run, and re-validate after any Composer upgrade. (b) False red from flake — network fetch, Packagist outage, or upstream HEAD drift; mitigate with pinned versions and a manual re-run policy that distinguishes reruns from investigation. (c) Environment mismatch — downstream suite needs services your runner lacks; exclude that target explicitly rather than letting it fail permanently. (d) Log opacity — install and test output interleave; enforce --no-colors in CI and verbose only on failure.
Ramp-up. Hours to first green run, roughly one to two engineer-days to a trustworthy required check, plus ongoing per-target maintenance measured in minutes per month. That is a cheap bet against building per-consumer bespoke jobs — appropriate given the 57.76 opportunity score and the fact that the alternative harness is a known-expensive custom pipeline. Cap the commitment: adopt as a CI convention with a documented exit, not as infrastructure.
How can I help you explore Laravel packages today?