Dissect Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require jakubledl/dissect
   

   Add to composer.json under require or require-dev depending on your use case.

2. First Use Case: Parse a simple arithmetic expression (e.g., "3 + 5 * 2"):

   use Dissect\Lexer\Lexer;
   use Dissect\Parser\Parser;
   
   $lexer = new Lexer();
   $tokens = $lexer->tokenize('3 + 5 * 2');
   
   $parser = new Parser();
   $ast = $parser->parse($tokens);
   

3. Where to Look First:

   • Documentation for lexer/parser concepts.
   • src/Dissect/Lexer/ for tokenization logic (e.g., Lexer.php, Token.php).
   • src/Dissect/Parser/ for parsing rules (e.g., Parser.php, Node.php).

Implementation Patterns

Core Workflows

1. Lexing:

   • Define custom token patterns by extending Lexer or implementing TokenStream:

     $lexer = new Lexer();
     $lexer->addPattern('/\d+/', 'NUMBER'); // Custom token type
     $tokens = $lexer->tokenize($input);
     

   • Use built-in patterns (e.g., IDENTIFIER, STRING, OPERATOR).

2. Parsing:

   • Parse tokens into an Abstract Syntax Tree (AST):

     $parser = new Parser();
     $ast = $parser->parse($tokens); // Returns a Node object
     

   • Extend Parser to support custom grammars (e.g., for domain-specific languages).

3. Integration with Laravel:

   • Validation: Use Dissect to validate user input (e.g., parse SQL-like queries safely).

     $lexer = new Lexer();
     $tokens = $lexer->tokenize(request('query'));
     if ($tokens->hasErrors()) {
         return back()->withErrors(['query' => 'Invalid syntax']);
     }
     

   • CLI Tools: Build custom artisan commands for parsing config files or scripts.

4. AST Traversal:

   • Walk the AST for transformations or analysis:

     $ast->accept(new class implements NodeVisitor {
         public function visit(Node $node) {
             // Custom logic (e.g., code generation, optimization)
         }
     });
     

Common Patterns

• Error Handling: Lexer/parser errors are thrown as exceptions. Catch LexerException or ParserException:

  try {
      $ast = $parser->parse($tokens);
  } catch (\Exception $e) {
      report($e); // Log via Laravel's error handler
  }
  

• Reusable Parsers: Encapsulate parsers in services:

  class QueryParserService {
      public function parse(string $query): Node {
          $lexer = new Lexer();
          $parser = new Parser();
          return $parser->parse($lexer->tokenize($query));
      }
  }
  

Gotchas and Tips

Pitfalls

1. Performance:

   • Lexing/parsing large inputs (e.g., multi-line SQL) can be slow. Optimize with:
     • Pre-compiled regex patterns (cache Lexer instances).
     • Lazy evaluation for streaming inputs.

2. Recursive Descent Limits:

   • The parser uses recursive descent, which may hit PHP’s stack limit for deeply nested structures. Mitigate by:
     • Using iterative parsers (e.g., Parser::parseIterative() if available).
     • Increasing xdebug.max_nesting_level (temporarily) during development.

3. Token Ambiguity:

   • Overlapping patterns (e.g., /=/ vs /==/) may cause unexpected tokens. Order matters:

     $lexer->addPattern('/==/', 'EQUALS');  // Must come before /=/
     $lexer->addPattern('/=/', 'ASSIGN');
     

4. AST Mutability:

   • Nodes are immutable by default. Clone or rebuild the AST for modifications:

     $newAst = clone $ast;
     $newAst->setChild(0, new NumberNode(42));
     

Debugging Tips

• Token Inspection: Dump tokens to debug lexing issues:

  foreach ($tokens as $token) {
      dump($token->getValue(), $token->getType());
  }
  

• Parser Debugging: Enable parser tracing (if supported) or log AST nodes:

  $ast->accept(new class implements NodeVisitor {
      public function visit(Node $node) {
          info("Node: {$node->getClass()}, Value: {$node->getValue()}");
      }
  });
  

Extension Points

1. Custom Lexers: Extend Lexer to support new languages:

   class CustomLexer extends Lexer {
       public function __construct() {
           $this->addPattern('/#include\s+(.*)/', 'INCLUDE');
       }
   }
   

2. Custom Parsers: Implement ParserInterface for domain-specific grammars:

   class JsonParser implements ParserInterface {
       public function parse(TokenStream $tokens) {
           // Custom parsing logic
       }
   }
   

3. Node Visitors: Extend functionality without modifying core classes:

   class CodeGeneratorVisitor implements NodeVisitor {
       public function visit(Node $node) {
           // Generate code from AST nodes
       }
   }
   

Laravel-Specific Quirks

• Service Container: Bind Dissect components as singletons for shared lexers/parsers:

  $this->app->singleton(Lexer::class, function () {
      return new Lexer();
  });
  

• Testing: Mock Lexer/Parser in unit tests:

  $mockLexer = Mockery::mock(Lexer::class);
  $mockLexer->shouldReceive('tokenize')->andReturn($tokens);