Dom Crawler Laravel Package

Getting Started

Install the component directly — it is not a Laravel package and there is nothing to register:

composer require symfony/dom-crawler:^7.4

Pick your constraint deliberately, because the parsing engine changed:

Also do not go below 7.4.12 / 8.0.12 / 6.4.40: CVE-2026-45071 (XXE in addXmlContent()) is fixed there, and 8.1.5/7.4.17/6.4.44 carry the selector and textarea/getLabel() correctness fixes.

The first use case in a Laravel codebase is almost never scraping — it is asserting structure in tests where assertSee() is too blunt:

// tests/Feature/InvoiceMailTest.php
use Symfony\Component\DomCrawler\Crawler;

$html = (new InvoiceMail($invoice))->render();
$crawler = new Crawler($html, 'https://app.test');

$this->assertSame('https://app.test/invoices/42', $crawler->filter('a.button')->attr('href'));
$this->assertCount(3, $crawler->filter('table tbody tr'));
$this->assertSame('€1,200.00', $crawler->filter('.total')->text());

Read the upstream component docs first (https://symfony.com/doc/current/components/dom_crawler.html) — the README is a stub. Then look at Crawler::filter(), Crawler::filterXPath(), and the Form/FormField classes; those three surfaces are 95% of day-to-day use.

Implementation Patterns

Fetch with Laravel, parse with DomCrawler. Do not pull in BrowserKit/Guzzle for HTTP. Http gives you timeouts, retries, and per-host rate limits that BrowserKit does not:

$response = Http::timeout(10)->retry(3, 200)
    ->withHeaders(['User-Agent' => config('scraper.ua')])
    ->get($url);

$crawler = new Crawler($response->body(), $url); // $uri resolves relative links later

Isolate it behind an application-owned boundary. Domain code should not import Symfony\Component\DomCrawler\*. Wrap it:

namespace App\Support\Html;

use Symfony\Component\DomCrawler\Crawler;

final class Document
{
    private Crawler $crawler;

    public function __construct(string $html, private ?string $baseUri = null)
    {
        $this->crawler = new Crawler($html, $baseUri);
    }

    /** @return list<string> */
    public function texts(string $selector): array
    {
        return $this->crawler->filter($selector)
            ->each(fn (Crawler $n) => trim($n->text()));
    }

    public function text(string $selector, ?string $default = null): ?string
    {
        $nodes = $this->crawler->filter($selector);
        return $nodes->count() ? trim($nodes->first()->text()) : $default;
    }

    public function attr(string $selector, string $attribute, ?string $default = null): ?string
    {
        $nodes = $this->crawler->filter($selector);
        if (!$nodes->count()) {
            return $default;
        }
        $node = $nodes->first();
        return $node->attr($attribute) ?? $default;
    }
}

That last method exists because attr() throws on a missing attribute. Every accessor returns null/[] instead of exploding, which is what a scraper actually wants.

Selectors: CSS first, XPath for the rest. filter() uses CssSelectorConverter — CSS3-ish, no jQuery extensions. There is no :contains(), :visible(), or :has(); use XPath:

// "link whose text mentions refund"
$crawler->filterXPath('//a[contains(normalize-space(.), "refund")]');

// parent/ancestor walking — CSS cannot do this
$crawler->filter('.price')->filterXPath('ancestor::section[1]//h2');

Rule of thumb: filter() for structure, filterXPath() for text-content predicates and ancestor axes.

Forms: use them to build requests, not to send them. form() gives you a value map; you hand it to Http:

$form = $crawler->selectButton('Sign in')->form();
$form['email'] = '[email protected]';
$form['password'] = 'secret';

$response = Http::asForm()->post($form->getUri(), $form->getPhpValues());

$form->getValues() is the cheapest way to assert CSRF tokens and hidden fields are present in a rendered Blade form. Checkboxes use ->tick()/->untick(); <select> uses ->select($value) (or ->select($label) for option text). Multi-selects take an array.

Links: resolve URLs, then enqueue. Relative hrefs resolve against the base URI you passed to the constructor:

$crawler->filter('a.next')->each(function (Crawler $node) use ($queue) {
    $queue->push($node->link()->getUri()); // absolute URL
});

Pagination/crawl loops belong on a queued job per page, with the discovered URLs dispatched as new jobs. Never recurse inside one process — you lose retries, backoff, and observability.

XML/RSS/Atom needs namespaces registered before XPath:

$crawler = new Crawler();
$crawler->addXmlContent($xml);
$crawler->registerNamespace('atom', 'http://www.w3.org/2005/Atom');
foreach ($crawler->filterXPath('//atom:entry') as $entry) { /* ... */ }

Use addXmlContent(), not addHtmlContent(), for feeds — HTML parsing mangles case-sensitive tags.

Gotchas and Tips

Almost every method throws on an empty node list. text(), html(), attr(), link(), form(), and eq($i) all raise InvalidArgumentException: The current node list is empty. (or an out-of-bounds variant). A page redesign then produces a stack trace instead of a null. Every extraction point needs a count() guard or an each() — the Document wrapper above is the cheap systemic fix.

filter() is scoped to the current nodes, including its XPath. filterXPath('//div') from a sub-crawler does not search the whole document; it is rewritten relative to the matched nodes, so you cannot climb back to an ancestor this way. Keep a reference to the root crawler if you need document-wide queries. This is also the area that was patched for combinator/positional-pseudo-class handling in 7.4.17/8.1.5 — if you are on an older patch, chained filter('.a > .b')->filter(':first-child') may silently misbehave.

Charset is the #1 silent corruption source. libxml assumes ISO-8859-1 unless told otherwise. Always pass the charset explicitly when you know it: $crawler->addHtmlContent($body, 'UTF-8'). Passing the charset in the Crawler constructor does not do this — the constructor's arguments are $node, $uri, $baseHref. Accented text turning into é is this bug, every time. Note the fix in 8.1.5 narrowed meta-tag charset sniffing; before that, a spurious mid-document <meta charset> could override your explicit value.

Parsing is lenient to the point of lying. libxml repairs broken markup without complaint — unclosed tags get closed, <table> gets a <tbody> injected, stray text gets hoisted out of the table. Assertions about DOM shape can pass locally and fail in production when a CMS changes a snippet. When a selector "should" match and does not, dump $crawler->html() (the serialized, post-repair tree) rather than the raw input — you are usually debugging libxml's repair, not your selector.

Memory and speed. The whole document is loaded into a DOMDocument; there is no streaming. A 5 MB page can cost tens of MB once parsed. For sitemap-scale XML, use XMLReader instead. For repeated selector evaluation, cache the Crawler per request and reuse it — re-parsing the same HTML is the common waste.

text() normalizes whitespace; html() does not. text() trims and collapses runs of whitespace, which is usually what you want for comparisons. When you need exact content (e.g. <pre> blocks or JSON in a <script>), use text(null, false) or $node->html(). html() on a multi-node crawler returns only the first node's inner HTML. Also note 8.1.5/7.4.17 fixed textarea reading — earlier versions returned truncated content for <textarea> with newlines.

XXE: pin the patch and stay off untrusted XML defaults. addXmlContent() historically enabled validateOnParse, which allowed external entity resolution. Fixed in 5.4.52/6.4.40/7.4.12/8.0.12, but if you parse XML from a third party, add LIBXML_NONET via the options argument and never call libxml_disable_entity_loader-era workarounds.

Do not use it for HTML sanitization. DomCrawler parses and navigates; it does not make markup safe. Cleaning user-submitted HTML is mews/purifier or HTMLPurifier, not this.

Extension points are thin — extend by composition. There is no manager, driver, or config. You cannot plugin a custom parser, and Crawler is final in v8. The right seam is your own Document/Selector service (as above), registered as a singleton in a service provider if you want a shared instance. If you need caching of fetched pages, put it in the acquisition layer (Http::withOptions + a cache middleware), never in the crawler.

Testing tip: $response->getContent() from Laravel's test client feeds DomCrawler perfectly, including assertSee-style checks that need structure:

$crawler = new Crawler($this->get('/pricing')->getContent());
$crawler->filter('.plan')->each(function (Crawler $plan) {
    $this->assertNotSame('', $plan->filter('.plan-price')->text());
});

Combine with assertSee for text presence and DomCrawler for shape, ordering, and attribute values — that is where the component pays for itself on day one.