Simplehtmldom Laravel Package

Getting Started

Minimal Steps

1. Installation Add the package via Composer:

   composer require weglot/simplehtmldom
   

   Require the autoloader in your project (Laravel handles this automatically).

2. First Use Case Parse a simple HTML string or fetch a URL:

   use SimpleHTMLDom\SimpleHTMLDom;
   
   // Parse a string
   $html = str_get_html('<div><h1>Hello</h1></div>');
   echo $html->find('h1', 0)->plaintext; // Output: "Hello"
   
   // Fetch a URL (requires allow_url_fopen or cURL)
   $html = file_get_html('https://example.com');
   echo $html->find('title', 0)->plaintext; // Output: "Example Domain"
   

3. Where to Look First

   • SimpleHTMLDom Documentation (mirrored in the package).
   • src/SimpleHTMLDom.php for core methods (e.g., find(), findAll(), plaintext).
   • Laravel-specific examples in the examples/ directory (if provided).

Implementation Patterns

Common Workflows

1. Scraping Dynamic Content Useful for parsing HTML responses from APIs or SPAs (e.g., extracting data from a rendered page):

   $html = file_get_html('https://example.com/api/rendered-page');
   $items = $html->find('.product-item');
   foreach ($items as $item) {
       $title = $item->find('.title', 0)->plaintext;
       $price = $item->find('.price', 0)->plaintext;
       // Store in DB or process further
   }
   

2. Modifying HTML Before Rendering Clean or transform HTML before passing it to Blade:

   $html = str_get_html($rawHtml);
   $html->find('.unwanted-class', 0)->outertext = ''; // Remove element
   $cleanHtml = $html->save();
   return view('template', ['content' => $cleanHtml]);
   

3. Integration with Laravel HTTP Client Fetch and parse HTML in a single request:

   use Illuminate\Support\Facades\Http;
   
   $response = Http::get('https://example.com');
   $html = str_get_html($response->body());
   $links = $html->find('a');
   

4. Extracting Structured Data Convert HTML to arrays for Eloquent models:

   $html = file_get_html('https://example.com/products');
   $products = [];
   foreach ($html->find('.product') as $product) {
       $products[] = [
           'name' => $product->find('.name', 0)->plaintext,
           'price' => $product->find('.price', 0)->plaintext,
       ];
   }
   Product::insert($products);
   

Tips for Laravel Integration

• Service Provider Binding: Bind the parser to Laravel’s container for dependency injection:

  $this->app->bind(SimpleHTMLDom::class, function () {
      return new SimpleHTMLDom();
  });
  

• Artisan Commands: Use for bulk scraping or maintenance tasks:

  use SimpleHTMLDom\SimpleHTMLDom;
  
  class ScrapeCommand extends Command {
      protected $signature = 'scrape:products';
      public function handle() {
          $html = file_get_html('https://example.com/products');
          // Process and save data
      }
  }
  

• Middleware for HTML Processing: Pre-process HTML in incoming requests (e.g., sanitization):

  public function handle($request, Closure $next) {
      $request->merge(['clean_html' => str_get_html($request->input('html'))]);
      return $next($request);
  }
  

Gotchas and Tips

Pitfalls

1. Memory Limits Parsing large HTML documents (e.g., full e-commerce pages) may hit PHP’s memory limit. Use str_get_html() with chunks or stream parsing:

   $html = file_get_html('https://example.com/large-page', false, null, 0, null, true);
   

   • Fix: Increase memory_limit in php.ini or process pages in batches.

2. Relative URLs Links/URLs in parsed HTML are relative by default. Resolve them with:

   $baseUrl = 'https://example.com';
   $links = $html->find('a');
   foreach ($links as $link) {
       $absoluteUrl = $baseUrl . $link->href;
   }
   

3. Encoding Issues HTML with non-ASCII characters (e.g., UTF-8) may break parsing. Ensure proper encoding:

   $html = str_get_html($rawHtml, true, false, 'UTF-8');
   

4. Dynamic Content SimpleHTMLDom cannot parse JavaScript-rendered content. Use tools like Puppeteer or Symfony Panther for SPAs.

5. Case Sensitivity CSS selectors are case-insensitive by default. For strict matching, use:

   $html->find('[class="ExactClass"]', 0);
   

Debugging

• Selector Not Working? Test selectors in browser DevTools first. Common issues:

  • Missing quotes in attributes: [href=example] → [href="example"].
  • Spaces in classes: .class1.class2 (correct) vs .class1 .class2 (incorrect).

• Empty Results? Check for:

  • Typos in selectors.
  • Hidden elements (e.g., display: none or visibility: hidden). Use $element->plaintext to verify.

• Performance Bottlenecks Cache parsed HTML or use Laravel’s cache:

  $html = Cache::remember('parsed_html', now()->addHours(1), function () {
      return file_get_html('https://example.com');
  });
  

Extension Points

1. Custom Selectors Extend functionality by adding methods to SimpleHTMLDom:

   SimpleHTMLDom::addMethod('findByDataAttr', function ($attr, $value) {
       return $this->find("[data-$attr='$value']");
   });
   

2. Laravel Events Trigger events when parsing specific elements:

   $html->find('.product')->each(function ($element) {
       event(new ProductScraped($element->plaintext));
   });
   

3. Integration with Laravel Collections Convert parsed elements to Collections for fluent querying:

   $collection = collect($html->find('.items'))
       ->map(fn ($item) => $item->plaintext)
       ->filter();
   

4. Testing Mock SimpleHTMLDom in tests:

   $html = str_get_html('<div><p>Test</p></div>');
   $this->assertEquals('Test', $html->find('p', 0)->plaintext);
   

Config Quirks

• Default Options: str_get_html() accepts these options:

  str_get_html($html, $lowercase, $forceTagsClosed, $target_charset, $stripRights, $defaultBRText);
  

  • Set $forceTagsClosed = true to auto-close tags (e.g., <div><p>Text</div> → <div><p>Text</p></div>).

• Disable HTML5 Parser: For strict HTML4 parsing, use:

  $html = str_get_html($html, false, false, 'UTF-8', false, false, '//');