Parser Utils Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require yosymfony/parser-utils
   

   Add to composer.json under require-dev if only for testing/parsing tools.

2. First Use Case: Parse a simple arithmetic expression (e.g., "3 + 5 * 2").

   use Yosymfony\ParserUtils\Parser;
   
   $parser = new Parser();
   $parser->addRule('expr', function($parser) {
       return $parser->exprAdd();
   });
   $parser->addRule('exprAdd', function($parser) {
       $left = $parser->exprMul();
       while ($parser->peek() === '+') {
           $parser->consume();
           $left += $parser->exprMul();
       }
       return $left;
   });
   $parser->addRule('exprMul', function($parser) {
       $left = $parser->number();
       while ($parser->peek() === '*') {
           $parser->consume();
           $left *= $parser->number();
       }
       return $left;
   });
   $parser->addRule('number', function($parser) {
       return (int)$parser->consume();
   });
   
   $result = $parser->parse('3 + 5 * 2'); // Returns 13
   

3. Where to Look First:

   • Source Code (if available) for rule methods (peek(), consume(), parse()).
   • Parser class docs (if any) for core methods like addRule(), parse(), and error handling.

Implementation Patterns

Recursive Descent Parsing Workflow

1. Define Grammar Rules: Break input into hierarchical rules (e.g., expr → exprAdd → exprMul → number).

   $parser->addRule('ruleName', function($parser) {
       // Recursive calls to sub-rules or terminal matches
   });
   

2. Terminal vs. Non-Terminal Rules:

   • Terminal: Matches literal values (e.g., number, identifier).

     $parser->addRule('identifier', function($parser) {
         return $parser->consume(); // Assumes input is alphanumeric
     });
     

   • Non-Terminal: Combines other rules (e.g., exprAdd combining exprMul with +).

3. Error Handling: Use Parser::expect() or custom logic to handle mismatches:

   try {
       $result = $parser->parse($input);
   } catch (\RuntimeException $e) {
       // Log or rethrow with context
   }
   

4. Integration with Laravel:

   • Validation: Parse user input (e.g., custom query syntax) before validation.

     $request->validate([
         'query' => 'required|string',
     ]);
     $parsed = $parser->parse($request->query);
     

   • Artisan Commands: Use for CLI argument parsing (e.g., --filter="type:active").
   • Service Providers: Bind Parser as a singleton for reuse:

     $this->app->singleton(Parser::class, function() {
         return new Parser();
     });
     

5. Performance Tips:

   • Pre-compile rules for complex grammars (cache Parser instance).
   • Avoid deep recursion for large inputs (use iterative approaches or tail-call optimization if possible).

Gotchas and Tips

Pitfalls

1. Left Recursion: The package doesn’t natively handle left recursion (e.g., expr → expr '+' term). Rewrite rules to avoid it:

   // Bad (left-recursive):
   $parser->addRule('expr', function($parser) {
       return $parser->expr() + $parser->term(); // Infinite loop
   });
   // Good (right-recursive):
   $parser->addRule('expr', function($parser) {
       $left = $parser->term();
       while ($parser->peek() === '+') {
           $parser->consume();
           $left += $parser->term();
       }
       return $left;
   });
   

2. Input Consumption:

   • consume() removes the matched token from the input stream. Over-consumption causes RuntimeException.
   • Use peek() to check ahead without consuming:

     if ($parser->peek() === '(') {
         $parser->consume(); // Safe to consume after peek
     }
     

3. Whitespace Handling: The parser treats whitespace as significant by default. Strip it manually:

   $input = preg_replace('/\s+/', '', $input);
   

4. No Built-in Tokenization: You must pre-process input into tokens or handle raw strings carefully:

   // Risky: Assumes input is tokenized
   $parser->addRule('number', function($parser) {
       return (int)$parser->consume(); // Fails on "123abc"
   });
   // Safer: Use regex or a tokenizer
   

Debugging Tips

1. Log Parser State: Add debug output to track input position and matched rules:

   $parser->addRule('debug', function($parser) {
       logger()->debug('Position:', $parser->getPosition());
       return null;
   });
   

2. Test Incrementally: Start with a single rule (e.g., number) and expand gradually. Use try-catch to isolate failures:

   try {
       $parser->parse('123');
   } catch (\RuntimeException $e) {
       dd($e->getMessage(), $parser->getPosition());
   }
   

3. Edge Cases:

   • Empty input: Wrap parse() in checks.
   • Malformed input: Use Parser::expect() or custom validation:

     if (!$parser->peek()) {
         throw new \InvalidArgumentException("Unexpected end of input");
     }
     

Extension Points

1. Custom Error Messages: Override default error messages by catching RuntimeException and rethrowing with context:

   try {
       $parser->parse($input);
   } catch (\RuntimeException $e) {
       throw new \RuntimeException("Parse error at position {$parser->getPosition()}: {$e->getMessage()}");
   }
   

2. Tokenization Layer: Extend the parser with a tokenizer for complex grammars (e.g., SQL-like queries):

   class QueryTokenizer {
       public function tokenize(string $input): array {
           // Implement lexing logic (e.g., split into [SELECT, FROM, users])
       }
   }
   

3. Laravel Service Integration: Create a facade or helper for common parsing tasks:

   // app/Helpers/ParserHelper.php
   class ParserHelper {
       public static function parseQuery(string $query) {
           $parser = app(Parser::class);
           // ... setup rules
           return $parser->parse($query);
       }
   }
   

4. Performance Optimization: For high-frequency parsing (e.g., API request validation), memoize Parser instances:

   $parserPool = [];
   function getParser(string $grammarName) {
       if (!isset($parserPool[$grammarName])) {
           $parserPool[$grammarName] = new Parser();
           // Setup rules...
       }
       return $parserPool[$grammarName];
   }