Xml String Streamer Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require prewk/xml-string-streamer
   

   Add to composer.json if using a monorepo or custom package management.

2. First Use Case: Streaming Large XML

   use Prewk\XmlStringStreamer\Streamer;
   
   $streamer = new Streamer();
   $streamer->streamFromFile('path/to/large.xml', function ($node) {
       // Process each node as it's encountered
       if ($node->name === 'record') {
           // Handle record data
       }
   });
   

3. Key Files to Review

   • src/Streamer.php – Core class with streaming logic.
   • tests/ – Example use cases (e.g., handling namespaces, attributes).
   • README.md – Quickstart and CLI examples.

Implementation Patterns

Workflows

1. Streaming from Files

   $streamer->streamFromFile('file.xml', function ($node) {
       // Node is a stdClass with properties: name, attributes, children, text
   });
   

2. Streaming from Strings

   $streamer->streamFromString($xmlString, function ($node) {
       // Process node
   });
   

3. Event-Based Processing

   $streamer->on('startElement', function ($name, $attrs) {
       // Handle element start
   });
   

4. Integration with Laravel

   • Queue Jobs for Large Files

     dispatch(new ProcessLargeXmlJob('file.xml'));
     

     class ProcessLargeXmlJob implements ShouldQueue {
         public function handle() {
             $streamer = new Streamer();
             $streamer->streamFromFile(storage_path('app/large.xml'), fn($node) => {
                 // Process and store in DB
             });
         }
     }
     

   • Laravel Service Provider Bind the streamer to the container for dependency injection:

     $this->app->singleton(Streamer::class, fn() => new Streamer());
     

5. Transforming XML to Collections

   $records = collect();
   $streamer->streamFromFile('data.xml', function ($node) use (&$records) {
       if ($node->name === 'user') {
           $records->push((object) [
               'id' => $node->attributes->id,
               'name' => $node->text,
           ]);
       }
   });
   

Gotchas and Tips

Pitfalls

1. Memory Leaks with Closures

   • Avoid capturing large objects in stream callbacks. Use use (&$var) sparingly.
   • Example of bad practice:

     $bigArray = range(1, 1_000_000);
     $streamer->streamFromFile('file.xml', function ($node) use ($bigArray) {
         // $bigArray is copied into closure scope; inefficient
     });
     

   • Fix: Pass only necessary data or use global/static storage.

2. Namespace Handling

   • The streamer does not automatically resolve namespaces. Explicitly check for prefixed tags:

     if ($node->name === '{http://example.com}record') {
         // Handle namespaced record
     }
     

3. Attribute Access

   • Attributes are returned as an object ($node->attributes). Use isset() or property_exists() to check for keys:

     if (property_exists($node->attributes, 'id')) {
         $id = $node->attributes->id;
     }
     

4. Encoding Issues

   • Large XML files may use non-UTF-8 encodings. Force UTF-8 if needed:

     $streamer->setEncoding('UTF-8');
     

5. Performance with Deeply Nested XML

   • For XML with extreme nesting (e.g., 100+ levels), consider flattening logic or using iterative processing:

     $streamer->streamFromFile('file.xml', function ($node) {
         if ($node->name === 'deeply_nested_tag') {
             // Process and reset state to avoid stack overflow
         }
     });
     

Debugging Tips

1. Log Node Structures

   $streamer->streamFromFile('file.xml', function ($node) {
       \Log::debug('Node:', [
           'name' => $node->name,
           'attrs' => (array) $node->attributes,
           'text' => substr($node->text, 0, 100), // Truncate long text
       ]);
   });
   

2. Validate XML First Use libxml_use_internal_errors() to catch malformed XML before streaming:

   libxml_use_internal_errors(true);
   $doc = simplexml_load_file('file.xml');
   if ($doc === false) {
       foreach (libxml_get_errors() as $error) {
           \Log::error($error->message);
       }
       throw new \RuntimeException('Invalid XML');
   }
   

3. Test with Small Files Start with a 10KB XML file to verify logic before scaling to GB-sized files.

Extension Points

1. Custom Node Processing Extend Streamer to add pre/post-processing hooks:

   class CustomStreamer extends Streamer {
       protected function beforeProcessNode($node) {
           // Modify node before callbacks
       }
   }
   

2. Plugin System Use Laravel’s service container to bind custom processors:

   $this->app->bind(Streamer::class, function ($app) {
       $streamer = new Streamer();
       $streamer->addProcessor(new CustomXmlProcessor());
       return $streamer;
   });
   

3. Event Dispatching Integrate with Laravel Events for cross-cutting concerns:

   $streamer->on('startElement', function ($name) {
       event(new XmlElementStarted($name));
   });