Multi Tester Laravel Package

Getting Started

Before anything else, confirm your package is actually consumable by other projects: you need a composer.json at the repo root with a "name" property (e.g. acme/my-package). multi-tester uses that value to know which folder to swap inside each downstream project's vendor/ directory. No name, no replacement.

Install it locally as a dev dependency:

composer require kylekatarnls/multi-tester --dev

That gives you vendor/bin/multi-tester. The first thing you'll do is scaffold a config file:

vendor/bin/multi-tester --add=acme/consumer-app

This creates .multi-tester.yml with a default entry. Open it and sanity-check the shape:

acme/consumer-app:
  clone: git clone https://github.com/acme/consumer-app.git .
  install: composer install
  script: vendor/bin/phpunit

Every key except the package name is optional. Omit clone and the tool queries Packagist for the Git URL. Omit install and you get composer install --no-interaction. Omit script and you get vendor/bin/phpunit --no-coverage. So the minimum viable entry is literally:

acme/consumer-app: default

Then run the full suite:

vendor/bin/multi-tester

Use -v / --verbose the first several times. You want to see what clone command it picked, which version of your package got swapped in, and which script ran. The "first use case" here is the classic one: you're about to tag a release of acme/my-package, and before you do, you want to know that acme/consumer-app still passes against your working tree — not against the last tagged version.

If you'd rather not touch composer.json, multi-tester also supports a global install, which is fine for a one-off audit.

Implementation Patterns

Version matrices as your support contract. The most valuable pattern is registering the same package at multiple versions so your support claims are tested, not asserted:

acme/consumer-app:
  version: ^2.0
acme/consumer-app:3.*: default
acme/consumer-app:4.*: default

The version string can go inline in the key (package:constraint) with a value of default meaning "all settings use defaults." Combine with semver strings in version: (>4.5, ~3.1.0, ^3.2). Without a version, the last stable is used. This is how you turn "we support Laravel 10 and 11" in your README into something a CI job can falsify.

Travis passthrough. If a downstream project has a .travis.yml, you don't have to reverse-engineer its install and test commands. Set either field to travis and multi-tester copies the corresponding command from that file:

acme/legacy-consumer:
  install: travis
  script: travis
# shorthand for both:
acme/legacy-consumer: travis

This is the pragmatic default when you're testing OSS packages you don't control, because their Travis config is the authoritative record of how they expect to be built.

A dedicated CI lane, not every build. The README's Travis recipe is the pattern to copy regardless of your CI vendor:

script:
  - if [ "$MULTITEST" != "on" ]; then vendor/bin/phpunit; fi;
  - if [ "$MULTITEST" = "on" ]; then vendor/bin/multi-tester; fi;

One matrix entry flips MULTITEST=on. Downstream suites are slow and noisy by nature (clone + full composer install per target), so keeping them off the critical path matters. GitHub Actions is also supported (as of 2.5.0) — the same idea applies with an env-gated step.

Keep the matrix fast and stable. Two config knobs do most of the work:

• stop_on_failure — bail on the first downstream failure. Use it locally and on PRs where you want signal fast; drop it on scheduled/nightly runs where you want the full report.
• success_only (2.0.0+) — pin each downstream project to its last successful commit rather than HEAD. This is the difference between "our change broke them" and "their repo was already red when we looked."

Config location and format are both flexible. The default is .multi-tester.yml in cwd. Pass a path as the first argument (vendor/bin/multi-tester ./ci/multitest.json) and multi-tester accepts either YAML or JSON. If your composer.json isn't adjacent to the config file, set the directory explicitly:

config:
  directory: ../foobar

Packages that replace other packages. If your package declares a "replace" key (common for forks and polyfills), multi-tester handles the autoload and vendor mapping as of 1.4.0, with improved support in 2.6.1. You generally don't need to do anything special, but be aware of it when the swap doesn't seem to take effect.

Useful flags in practice: q / --quiet-install (1.3.0) suppresses the barrage of Composer output while still surfacing test results — worth turning on for any CI log you actually read. --colors / --no-colors (2.5.0) overrides auto-detection for CI environments that don't advertise TTY support.

Gotchas and Tips

A green result is only as meaningful as the downstream test suite. This is the single biggest trap. multi-tester will happily report success against a consumer with no tests, or tests that don't exercise your package. Before you trust it, verify that (a) the downstream suite actually fails when you intentionally break your package, and (b) the package you're testing is genuinely in their dependency tree and not shadowed by a transitive constraint. Zero dependents in a repo's metadata is usually an honest signal.

Network access is a hard requirement. Every run clones from GitHub (or the VCS URL Packagist returns) and runs composer install. Air-gapped runners, aggressive egress policies, or rate-limited CI will break here. Budget for a clone-and-install per target per run; if that's expensive, use success_only and stop_on_failure to cut work, and consider running the lane on a schedule rather than per-push.

The replacement is mechanical, and it can miss. multi-tester swaps your package directory inside vendor/ based on the "name" from composer.json. If the downstream project pins your package through a replace or a path repository or a lockfile that resolves oddly, the swap may not take effect the way you expect. Debug with -v, which since 2.5.0 prints substantially more detail — check the version resolved and confirm the code being tested is your working copy.

Composer version drift. Composer 2 is supported and default (2.4.0), and you can pin a specific Composer version in config (2.2.0). If a downstream project's composer.json needs a plugin that prompts for allow-plugins, 2.6.1 added a shortcut for registering that — worth knowing when a consumer install hangs on an interactive prompt. install: composer install --no-interaction is the default for a reason, but a custom install: string is on you to make non-interactive.

Quiet the installs, not the tests. -q / --quiet-install applies to all install output (fixed in 1.3.1) but leaves test output intact. Don't conflate it with a general quiet mode — you still want to see failures.

Packagist is not infallible. If you omit clone, the Git URL comes from Packagist's API; since 2.3.0 it falls back to libraries.io when Packagist fails. When both are unavailable you'll see a raised exception rather than a silent failure (2.5.2 tightened this). If you're behind a proxy or Packagist is degraded and you don't want to depend on it at test time, always specify clone: explicitly — it's the most robust configuration.

Don't over-interpret a "null conclusion." Since 2.5.1 a null conclusion is treated as success, and older releases had looser JSON validation (2.2.1, 2.2.2 tightened this). On very old versions you could see false greens when the API returned something unexpected. Pin to a 2.5.x+ release if you care about accurate failure reporting.

Working-directory handling is subtle. Multi-tester creates its working directory inside the current directory (0.1.6) and aligns directories across processes (0.1.8). If you're invoking it from a wrapper script, cd into the intended root first, or set config.directory explicitly. Mis-set directories produce confusing "package not found in vendor" symptoms rather than a clean error.

PHP support floor: 2.6.0 dropped PHP 7.1, so you need PHP 7.2+ (and Symfony 8 is permitted as of the same release). If you're maintaining a 7.1-targeting project, pin multi-tester to a 2.5.x release.

It is not a test framework. It runs whatever script you hand it. Downstream suites needing databases, queues, browsers, or service containers require their own harness — multi-tester will just be a fragile shell wrapper around whatever you already have. Don't try to make it an E2E orchestrator; use it to run suites that can stand alone, and reach for proper contract/E2E tooling for the rest.

Extension points are thin, by design. There's no plugin system; the config file is the API. If you need conditional behavior (e.g. "only run this downstream project on main"), solve it at the CI layer with env-gated steps and separate config files rather than trying to make the YAML dynamic.