Wikipeg Laravel Package

Getting Started

WikiPEG is a PEG parser generator for PHP: you write a .pegjs grammar whose semantic actions are PHP, and a Node-based compiler (ported from PEG.js 0.10) emits a PHP parser class extending WikiPEG\Parser.

composer require wikimedia/wikipeg   # runtime base class is needed in prod too
node vendor/wikimedia/wikipeg/bin/wikipeg --help   # flags mirror PEG.js 0.10 (-o/--output, --trace, ...)

First place to look: the package README, then vendor/wikimedia/wikipeg/bin/wikipeg (the CLI) and the runtime classes under vendor/wikimedia/wikipeg/src/ (Parser, SyntaxError, Expectation, Location). The generated file is the only thing you need to read carefully at runtime — it shows exactly which base-class helpers (text(), location(), error(), expected()) are available to your actions.

A minimal first use case — parsing a tiny key=value; argument list:

// grammar/Params.pegjs
start  = _ pairs:pair* { return $pairs; }
pair   = k:key "=" v:value ";" _ { return [ 'key' => $k, 'value' => $v ]; }
key    = $[a-zA-Z_] [a-zA-Z0-9_]* { return $text; }

node vendor/wikimedia/wikipeg/bin/wikipeg \
  --output src/Parser/ParamsParser.php \
  grammar/Params.pegjs

$ast = (new ParamsParser())->parse("foo=bar;baz=qux;");

Regenerate, require the file (or autoload it), parse. That is the whole loop.

Implementation Patterns

Keep the generated parser behind a facade. Generated code is machine output; don't let callers depend on its class. Wrap it:

final class TemplateArgsParser
{
    private ParamsParser $parser;

    public function __construct() { $this->parser = new ParamsParser(); }

    public function parse(string $source): TemplateArgs
    {
        try {
            return TemplateArgs::fromArray($this->parser->parse($source));
        } catch (SyntaxError $e) {
            throw new InvalidTemplateArgs($source, $e);  // keep the original
        }
    }
}

That wrapper is where you convert PEG output (nested arrays) into value objects, and where you attach line/column info from location() so downstream error messages point at the right place instead of at the AST.

Return spans, not just values. Any grammar that feeds an editor, linter, or error reporter should include location() in each node:

pair = k:key "=" v:value ";" { return [ 'key' => $k, 'value' => $v, 'loc' => $location ]; }

Offsets are character-based; PHP string functions are byte-based. Store offsets, but compute line/column for display and never slice the source with substr() using an offset without going through mb_substr().

Use the initializer and options for injection. The block at the top of the grammar becomes members of the generated class, so shared helper methods live there. $options (the second argument to parse()) is where you thread configuration — a strictness flag, a callback for namespace resolution, a startRule selector for parsing fragments of a larger grammar. This is the cleanest way to keep grammars pure while still letting the application customise behaviour.

Treat codegen as a build step, not a manual one. Add a Composer script:

"scripts": {
  "build-parser": "node vendor/wikimedia/wikipeg/bin/wikipeg --output src/Parser/ParamsParser.php grammar/Params.pegjs"
}

Run it in CI and follow with git diff --exit-code so a grammar edit that wasn't regenerated fails the build. Commit the generated file (it's small enough to live in src/) and mark it linguist-generated in .gitattributes — hand-editing generated output is the number one way to lose an afternoon.

Test the grammar, not the generator. PHPUnit data providers work well:

/** @dataProvider provideInputs */
public function testParse(string $input, array $expected): void
{
    $this->assertSame($expected, (new ParamsParser())->parse($input));
}

Keep a tests/grammar/ corpus of inputs paired with expected ASTs. Add a small set of negative cases asserting that SyntaxError is thrown with the expected location, since that is what users actually see.

Gotchas and Tips

• Generated/runtime version coupling. The emitted PHP and the WikiPEG\Parser base class are a matched pair. Pin the Composer version and regenerate after every upgrade; a stale generated file against a newer runtime is a subtle, confusing failure.
• Node is required at build time only. Production needs just the PHP runtime. If CI doesn't have Node, the build step silently gets skipped and you ship a stale parser — assert on the Node version explicitly.
• No left recursion. expr = expr "+" term / term will fail or blow the stack. Rewrite as a flat repetition and fold: term (op term)*.
• Empty-match loops hang. Any rule* where rule can match the empty string produces an infinite loop. Audit nullable rules and make them non-nullable or guard with a lookahead.
• No implicit whitespace. PEG doesn't skip anything for you. Define _ and __ rules and sprinkle them deliberately; forgetting one at the end of a top-level rule is the classic "works on one line, fails on two" bug.
• Ordered choice is greedy and committed only locally. Put specific alternatives before general ones ("null" / ident), and watch rules like (a / ab)* for exponential backtracking on malformed input. The cache/optimize option trades memory for time — measure before enabling on large inputs.
• PHP inside actions is still inside a grammar. Braces must balance, and the compiler's brace matching doesn't understand heredocs or complex interpolation. Keep actions to one or two statements; push real logic into initializer helpers or the wrapper class. Watch $ and backslashes when the grammar is embedded in another templating layer.
• Errors are noisy by default. WikiPEG\SyntaxError exposes location, expected, and found; the expected list often contains internal rule names. Either use --trace while developing, or map the error in your wrapper to something a human can act on. Don't surface raw expectation lists to end users.
• Extension points are limited by design. There's no plugin API in the generated parser — the initializer block and the options array are your hooks. If you find yourself wanting to override a generated method, the right move is almost always to add a labelled rule and do the work in the action.
• Debugging tip: to inspect a subrule, temporarily change its action to { throw new \RuntimeException(var_export([$a, $b], true)); }, run your failing test, and read the trace. Faster than guessing, and it disappears when you regenerate.