Stream Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require jstewmc/stream
   

   No additional configuration is required—just autoload the package.

2. First Use Case: Streaming a Large File

   use JSTewMc\Stream\Stream;
   
   $filePath = storage_path('app/large-file.txt');
   $stream = new Stream($filePath);
   
   while ($stream->valid()) {
       $char = $stream->current();
       // Process character (e.g., log, analyze, or emit)
       $stream->next();
   }
   

3. First Use Case: Streaming a String

   $string = "Hello, 世界!";
   $stream = new Stream($string);
   
   while ($stream->valid()) {
       $char = $stream->current();
       // Process character
       $stream->next();
   }
   

4. Where to Look First

   • Class Docs: Focus on JSTewMc\Stream\Stream methods (valid(), current(), next(), key(), rewind()).
   • Tests: Check the package’s test suite (if available) for edge cases (e.g., multi-byte characters, empty inputs).

Implementation Patterns

Core Workflows

1. File Streaming (Chunked Processing)

   $stream = new Stream(storage_path('app/logs/big.log'));
   $lineBuffer = '';
   
   while ($stream->valid()) {
       $char = $stream->current();
       if ($char === "\n") {
           // Process $lineBuffer (e.g., parse, store, or emit)
           $lineBuffer = '';
       } else {
           $lineBuffer .= $char;
       }
       $stream->next();
   }
   

2. String Transformation

   $stream = new Stream("Input String");
   $output = '';
   
   while ($stream->valid()) {
       $char = strtoupper($stream->current()); // Example transformation
       $output .= $char;
       $stream->next();
   }
   

3. Integration with Laravel Events/Jobs

   // In a job or event listener:
   $stream = new Stream($filePath);
   while ($stream->valid()) {
       dispatch(new ProcessCharacterJob($stream->current()));
       $stream->next();
   }
   

4. Memory-Efficient Pagination

   $stream = new Stream($filePath);
   $batchSize = 1000;
   $batch = [];
   
   while ($stream->valid()) {
       $batch[] = $stream->current();
       $stream->next();
   
       if (count($batch) >= $batchSize) {
           Process::chunk($batch, 100); // Laravel's chunk processing
           $batch = [];
       }
   }
   

Advanced Patterns

1. Custom Iterators Extend Stream or wrap it in a custom iterator for domain-specific logic:

   class LineStream extends Stream {
       public function __construct($source) {
           parent::__construct($source);
           $this->buffer = '';
       }
   
       public function next() {
           if ($this->current() === "\n") {
               $this->buffer = '';
           } else {
               $this->buffer .= $this->current();
           }
           parent::next();
       }
   
       public function getLine() {
           return $this->buffer;
       }
   }
   

2. Resource Cleanup Ensure streams are closed (especially for files) using PHP’s finally or Laravel’s try-catch-finally:

   $stream = new Stream($filePath);
   try {
       while ($stream->valid()) { /* ... */ }
   } finally {
       if ($stream instanceof \SplFileObject) {
           $stream->close();
       }
   }
   

Gotchas and Tips

Pitfalls

1. Multi-Byte Character Handling

   • The package claims multi-byte safety, but test with UTF-8 strings (e.g., "你好"). Edge cases may arise with surrogate pairs or incomplete sequences.
   • Fix: Validate input encoding or use mb_* functions for critical operations.

2. File Pointers and rewind()

   • Calling rewind() on a file stream resets the file pointer but does not reset memory buffers in custom iterators.
   • Tip: Reinitialize the Stream object if you need a fresh read.

3. Resource Leaks

   • File streams (SplFileObject) may leak if not closed. Use finally blocks or Laravel’s withFile() helper:

     withFile($filePath, function ($file) {
         $stream = new Stream($file);
         // Process...
     }); // Auto-closes file
     

4. Performance with Large Files

   • Streaming is memory-efficient, but I/O-bound operations (e.g., DB writes per character) will bottleneck.
   • Tip: Batch writes (e.g., collect 1000 characters, then flush to DB).

5. Empty Inputs

   • Stream may throw exceptions or behave unexpectedly with empty strings/files.
   • Tip: Add guards:

     if (empty($source)) {
         return; // or throw new \InvalidArgumentException();
     }
     

Debugging Tips

1. Inspect Stream State Use var_dump($stream->key()) to track position (useful for debugging infinite loops).

2. Log Characters For troubleshooting, log characters with their byte values:

   $char = $stream->current();
   \Log::debug("Char: {$char} | Bytes: " . bin2hex($char));
   

3. Test Edge Cases

   • Empty strings/files.
   • Files with BOM (Byte Order Mark) or mixed encodings.
   • Very large files (>1GB) to test memory usage.

Extension Points

1. Custom Stream Sources Extend Stream to support new sources (e.g., HTTP streams, database cursors):

   class HttpStream extends Stream {
       protected $handle;
   
       public function __construct($url) {
           $this->handle = fopen($url, 'r');
           parent::__construct($this->handle);
       }
   
       public function __destruct() {
           fclose($this->handle);
       }
   }
   

2. Lazy Evaluation Combine with Laravel’s LazyCollection for functional-style processing:

   $stream = new Stream($filePath);
   LazyCollection::make(function () use ($stream) {
       while ($stream->valid()) {
           yield $stream->current();
           $stream->next();
       }
   })->filter(fn($char) => $char !== ' ')->take(100);
   

3. Progress Tracking Wrap the stream to add progress callbacks:

   class ProgressStream {
       public function __construct(private Stream $stream, private callable $callback) {}
   
       public function next() {
           $this->callback($this->stream->key());
           $this->stream->next();
       }
   }