Dom Crawler Laravel Package

Product Decisions This Supports

• Build vs. buy for HTML/XML extraction. This is a library decision, not a platform decision: MIT-licensed, no seats, no vendor contract, no infrastructure. It collapses the "do we buy a scraping/parsing service?" question for a large class of features — internal ingestion, feed parsing, markup normalization — into a small Composer dependency plus application code. Default position should be build, with the buy trigger being JavaScript-rendered sources or sustained high-volume crawling.
• Roadmap scoping for data-ingestion features. Concrete use cases this unblocks: competitor/pricing and catalog monitoring, supplier or partner feed normalization (XML and HTML), content migration and import from legacy sites, parsing inbound HTML email bodies into structured records, and enriching CRM/records from public web pages. Each of these becomes a candidate epic rather than a "needs research" placeholder.
• QA and release-confidence investment. Via Symfony BrowserKit it enables functional tests that drive real forms and links — assert that our own rendered pages expose the right labels, field names, and link targets. That is a regression-safety purchase, not a feature: it protects conversion-critical flows (checkout, signup, lead capture) from silent markup breakage.
• Platform floor decision. Symfony DomCrawler 8.x requires PHP ≥ 8.4 and always parses per HTML5 rules; 7.x uses libxml. This turns an innocuous dependency into a PHP upgrade forcing function. The PM decision is sequencing: adopt 7.x now and defer the parse-fidelity change, or fold the PHP 8.4 floor into the next platform uplift. Do not let this arrive as a surprise mid-sprint.
• Ongoing ownership commitment. The library is cheap; the pipelines are not. Every adopted source needs a named owner, selector-drift alerts, and a refresh cadence. Approving the dependency without approving the operational budget is the failure mode — treat extraction reliability as a monitored product surface, not a background job.
• Competitive and market intelligence. Low-cost, low-risk capability for continuously tracking public pricing pages, changelogs, and job postings — useful input to pricing and positioning decisions without procurement or a data vendor.

When to Consider This Package

Adopt when:

• The source markup is server-rendered HTML or XML and reasonably well-formed — CSS/XPath selection gets you 90% of the value with a few lines of code.
• You need form and link traversal, not just extraction (submitting a search form, following pagination) — this is where it meaningfully beats hand-rolled DOMDocument code.
• You're already on Laravel/Symfony and resolving symfony/* transitively; the marginal dependency cost is effectively zero.
• You need it for test assertions on your own rendered output — high value, near-zero risk, and a good first slice to prove the pattern.
• Volume is modest to moderate (hundreds to low thousands of pages/day) and can live on queues behind Laravel's HTTP client.

Look elsewhere when:

• The target requires JavaScript execution to render content. DomCrawler does not run a browser; this is a hard stop. Escalate to a headless-browser approach (Laravel Dusk / Puppeteer-class tooling) or a managed scraping vendor.
• You need crawl orchestration at scale — scheduling, proxy rotation, CAPTCHA handling, per-host politeness, change detection. DomCrawler is the parser, not the crawler. Buying or building that layer is a separate decision.
• The need is one-off and manual — a spreadsheet and a browser extension may be the cheaper answer. Don't stand up a pipeline to solve a five-minute task.
• The markup you're parsing is your own and under your control — fix the source or expose structured data (JSON) instead of adding a parsing layer.
• PHP < 8.4 with no near-term upgrade path and you want v8 semantics — plan for 7.x behavior differences instead, or defer.

How to Pitch It (Stakeholders)

For executives (one paragraph): This is a free, MIT-licensed, widely-adopted PHP component that lets our engineers read and extract data from HTML and XML pages reliably. It turns several "someday" ideas — supplier feed ingestion, content migration, competitive price monitoring, and automated checks that our own checkout and signup forms still work — into things we can ship in a sprint instead of a quarter, with no vendor contract, no new infrastructure, and no per-seat cost. The realistic investment is not the library; it's one small, well-scoped pipeline with a named owner and monitoring, so it doesn't quietly rot. The main constraint to be aware of is a PHP 8.4 requirement for the newest major version, which we should sequence deliberately against our platform upgrade rather than absorb reactively.

For engineering (bullet summary):

• Stateless parsing primitive — no service provider, no config, no framework coupling. Already reachable transitively via Composer.
• Isolate it. Wrap behind an application-owned interface (e.g. App\Support\Html\DocumentParser) so domain and job code never imports Symfony\Component\DomCrawler\*. That boundary is what keeps exit cost low and lets us swap parsers if semantics shift.
• Respect the v7 → v8 fork. v8 requires PHP ≥ 8.4 and always parses per HTML5 rules; pre-8.0 uses libxml. Selector results on malformed markup will differ across that line — pin the major version deliberately and re-baseline fixtures when crossing it.
• Treat selector semantics as a moving target. Recent patch lines fixed matches(), closest(), children() for combinators, charset detection from meta tags, textarea content, and FormField::getLabel() quoting. Pin versions and keep golden-file tests on representative fixtures.
• Security: track the XXE fix. CVE-2026-45071 (XXE via addXmlContent()) is patched across 5.4/6.4/7.4/8.x lines. Confirm our pinned version is on a patched release before parsing any untrusted XML.
• The real work is around the library: HTTP timeouts/retries and conditional requests, per-host rate limiting, queueing, deduplication and change detection, persistence, and observability (parse failure rate, selector hit rate, drift alerts).
• Sizing: one sprint for two engineers to land a thin vertical slice end-to-end, plus a named owner for pipeline health thereafter. Do not size this as "add a dependency."