Wikipeg Laravel Package

Technical Evaluation

Architecture fit Wikipeg is a PEG (Parsing Expression Grammar) parser generator written in PHP, a port of Ian Piumarta's peg/leg. It sits at the build-time layer of an application rather than the runtime layer: you author .peg grammar files, run the CLI compiler, and it emits standalone PHP parser classes that you then commit or autoload. For a Laravel codebase this means it does not integrate with the framework's service container, facades, config system, or Eloquent at all — it is a pure library plus a console entry point. That is actually a clean architectural boundary: the generated parser is a plain PHP class, so it can be wrapped in a Laravel service provider or a dedicated app/Parsing namespace with minimal coupling. The realistic use case is narrow: you have a domain-specific language (DSL), template syntax, query language, config format, or markup dialect that regex and Laravel's Str helpers cannot responsibly handle. The package's provenance matters here — it is the parser generator behind Parsoid's wikitext grammar, which is a strong credibility signal for correctness on complex, ambiguous grammars. It is a poor fit if the actual requirement is JSON/YAML/CSV parsing (use symfony/* components) or HTML sanitization (use an existing sanitizer).

Integration feasibility Composer installation is straightforward — the package is a standard PSR-4 PHP library with a CLI binary, MIT licensed, no exotic extensions or native compilation. Two integration shapes exist:

1. Dev-dependency + committed artifact: add wikimedia/wikipeg to require-dev, keep grammar sources in resources/grammars/, generate parser classes into app/Generated/ (or similar), commit the output, and expose a Composer script or Artisan command to regenerate. CI fails if generated output drifts from grammar source.
2. Runtime dependency: install in require and generate at deploy time. This is riskier — it puts a compiler in the production image and makes deploys dependent on generation succeeding.

Shape 1 is the correct default for Laravel. The generated code is plain PHP with no runtime dependency on wikipeg itself, which means the compiled parser carries zero operational footprint.

Technical risk Several concrete risks stand out:

• Distribution governance. The GitHub repository is explicitly a mirror of gerrit.wikimedia.org/g/wikipeg. Issues and pull requests on GitHub are not the contribution path. As a TPM you must assume: no inbound PR-based bug fixes, no GitHub issue triage, and a contributor workflow (Gerrit/Gitiles) your team is not set up for. Any patch you need, you own locally as a fork or a patches/ entry via cweagans/composer-patches.
• Low adoption signal. 9 stars and 0 dependents is a genuine signal — you will be among the only teams using it outside Wikimedia's own stack. There is no Stack Overflow corpus, no Laravel ecosystem precedent, and no third-party tutorials. Debugging is source-reading.
• Generated-code PHP compatibility. Generated parsers are emitted PHP source; the emitter's output style may lag modern PHP versions (typed properties, readonly, PHP 8.3/8.4 deprecations). A PHP minor upgrade could produce deprecation noise in generated files that you cannot cleanly patch upstream. Plan to pin PHP versions and validate generation output in CI on every runtime upgrade.
• Grammar authoring skill gap. PEG is a distinct mental model — ordered choice, no left recursion (or restricted handling), greedy repetition, silent failures. Expect a real learning curve before the first correct grammar lands.
• Abandonment risk is low, evolution risk is real. Wikimedia depends on it via Parsoid, so it will not vanish, but its roadmap is driven entirely by Parsoid's needs, not yours.

Key questions

• What is the concrete parsing problem, and has a hand-written recursive-descent parser or an existing library been ruled out with evidence?
• Who on the team can own grammar authorship and debugging — is that a named person or an aspiration?
• Do we accept a fork-and-patch maintenance posture for a mirrored repository?
• Is generated parser output committed (recommended) or built at deploy time?
• What is the input size and throughput profile, and has the generated parser been benchmarked against the current approach?
• Are there license/attribution obligations from grammar files inherited from MediaWiki/Parsoid that need review?

Integration Approach

Stack fit Wikipeg fits Laravel as a compiler toolchain, not a framework component. The clean layout:

• resources/grammars/*.peg — grammar sources, reviewed like code.
• app/Parsing/Generated/*.php — emitted parser classes, committed, excluded from coverage and static analysis baselines (or included with a baseline).
• app/Parsing/<Domain>Parser.php — a thin hand-written wrapper that owns the public API (parse(string $input): AstNode), translates wikipeg's generic parse tree or visitor callbacks into your domain AST, and throws domain-specific exceptions.
• A ParserGenerateCommand Artisan command or a composer generate-parsers script that shells out to the wikipeg CLI.
• A single service-provider binding so the wrapper is injectable and mockable in tests.

This keeps wikipeg at the edge of the system. Nothing else in the application should use Wikimedia\WikiPEG\... — only the generate command and the wrapper. Enforce that with a PHPStan/PHPat architectural rule.

Migration path Treat this as greenfield rather than migration: wikipeg should be introduced for a new grammar-backed capability, or to replace an existing regex/fragile parser behind an unchanged interface. The migration sequence:

1. Establish the current parser's behavior contract with a characterization test corpus (input → expected output pairs) before touching anything.
2. Author the .peg grammar against that corpus, iteratively, with the generator in CI.
3. Run both parsers in parallel behind a feature flag, diffing outputs on real production inputs and logging mismatches.
4. Cut over when the diff rate is zero over a defined observation window; keep the flag for a rollback period.

Because the generated parser is a plain class, rollback is a config flip, not a deploy.

Compatibility

• Verify the package's declared PHP constraint against your runtime (composer why-not / composer prohibits). Do this early — a constraint conflict with an existing Laravel-adjacent dependency is a common dead end.
• Confirm the generated code passes your CI under your target PHP version and the next one you plan to adopt (pre-empt the deprecation risk).
• Autoloading: generated classes need PSR-4 mapping or a classmap entry; decide before generation so output paths are stable.
• Ensure the CLI binary works in your CI image (it is a Composer bin, so vendor/bin/).

Sequencing

1. Spike (1–2 weeks): one engineer authors a small grammar end-to-end, measures output quality and performance, and writes up the ergonomics. This is the go/no-go gate.
2. Governance decision: fork-and-patch policy, Gerrit access vs. local fork, PHP version pinning, generated-code review policy.
3. Toolchain: Artisan generate command, CI drift check, architectural rule confining wikipeg usage.
4. Feature implementation: grammar + wrapper + characterization corpus.
5. Shadow deployment and cutover behind a flag.
6. Runbook and handover documenting regeneration, grammar style guide, and debugging techniques.

Operational Impact

Maintenance The steady-state maintenance burden is concentrated in two places. First, the grammar itself — grammars accumulate edge cases and are easy to break subtly, so every grammar change needs a regression corpus and a diff review. Second, the toolchain — the generate command and CI drift check become build-critical infrastructure; if they break, no one can modify the parser. Budget a named owner. Because the GitHub repo is a mirror, upstream fixes require monitoring the Gerrit/Gitiles feed or the Parsoid repository's dependency bumps rather than watching a GitHub releases page. Recommend pinning an exact version in composer.json (no ^) and bumping deliberately, with the generated-output diff reviewed as part of the bump PR.

Support There is effectively no external support channel. No Laravel community precedent, negligible GitHub issue activity, and the official contribution path is Gerrit — which most PHP teams are not fluent in. Internal support is the only support: one or two engineers who understand PEG semantics and can debug parse failures. This is a bus-factor risk and should be documented as such. The upside is that the dependency surface is small and the generated code is readable PHP, so a competent engineer can always read the output.

Scaling Parsing is CPU-bound and synchronous. A PEG parser is generally linear-ish in input size but ordered choice means poorly ordered rules can be catastrophically slow on pathological inputs — this is the classic failure mode and it must be tested with adversarial inputs, not just happy-path fixtures. In a Laravel request path, treat parsing as something you bound: enforce a maximum input size, set an execution time budget (e.g., set_time_limit guard in a queued job), and push large documents to a queue worker rather than parsing inline in a web request. Memory is typically fine for moderate documents but grows with document size and backtracking depth; load-test with the largest real inputs, not synthetic averages. Horizontal scaling is trivial — the parser is stateless and per-process, so it scales with your existing PHP-FPM/queue workers. No new infrastructure.

Failure modes

• Silent parse degradation. PEG parsers can succeed while producing a structurally wrong tree when ordered choice picks an unintended alternative. Mitigation: schema-validate the resulting AST in the wrapper and reject anomalies rather than passing them downstream.
• Pathological backtracking / CPU exhaustion. A crafted input spikes CPU. Mitigation: input size caps, timeouts, queue isolation, and a fuzz corpus in CI.
• Generation drift. Someone edits a generated file or forgets to regenerate. Mitigation: CI check that regenerates and fails on diff.
• PHP upgrade breakage. Deprecations or syntax changes in generated output. Mitigation: generate-and-lint in CI under both current and next PHP versions.
• Upstream divergence. A needed fix exists only on Gerrit or lands in a later Parsoid bump. Mitigation: documented local patch policy with composer-patches and a scheduled review of that patch list.
• Owner departure. Single-person knowledge. Mitigation: mandatory grammar style guide, pair review on grammar changes, and at least two engineers able to regenerate and debug.

Ramp-up Expect two to four weeks before a competent PHP engineer is productive in wikipeg — roughly one week to internalize PEG semantics and the tool's visitor API, and another one to three weeks to author a non-trivial grammar against a real corpus. This is the largest hidden cost in the assessment and should be scheduled explicitly rather than absorbed as "learning on the job." Mitigate by scoping the first grammar deliberately small (a single well-bounded format, not the full domain language) so the team earns the toolchain and the debugging instincts on a low-stakes surface before committing to anything load-bearing. Given the low ecosystem adoption and mirror-only distribution, the recommendation is to proceed only if the parsing problem is genuinely non-trivial and the team can commit a named owner plus a fork-and-patch posture; otherwise a hand-written recursive-descent parser is likely cheaper to own over a multi-year horizon.