Weave Code
Dom Crawler Laravel Package
symfony/dom-crawler
Symfony DomCrawler makes it easy to navigate and query HTML/XML DOMs using CSS selectors and XPath. Extract links, forms, and text, filter nodes, and chain queries for robust scraping, testing, and content parsing in PHP.
Eases DOM navigation for HTML and XML documents
Frequently asked questions about Dom Crawler
- How do I install Symfony DomCrawler in a Laravel 10+ project?
- Run `composer require symfony/dom-crawler:^8.0` to install the latest version. Laravel’s autoloader will handle dependencies automatically. For PHP 8.1+ Laravel 9.x, use `symfony/dom-crawler:^7.4`. Ensure your `composer.json` aligns with your Laravel version’s PHP requirements.
- Can DomCrawler parse dynamically rendered JavaScript content in Laravel?
- No, DomCrawler only processes static HTML/XML. For JavaScript-rendered content, pair it with Symfony Panther (headless browser) or tools like Puppeteer. Use DomCrawler for post-rendered markup or static pages.
- What’s the best way to integrate DomCrawler with Laravel’s service container?
- Bind the Crawler as a singleton in `AppServiceProvider` using `app()->bind(Crawler::class, fn() => new Crawler());`. Then inject `Crawler` into controllers or services via constructor dependency injection. This ensures reuse across your application.
- Does DomCrawler support Laravel’s queue system for background scraping?
- Yes. Dispatch parsing jobs to Laravel Queues by wrapping Crawler logic in a job class (e.g., `ParseHtmlJob`). Use `dispatch(new ParseHtmlJob($html))` to process large datasets asynchronously without blocking requests.
- How do I handle malformed HTML in Laravel with DomCrawler?
- DomCrawler v8.0+ uses PHP 8.4’s native HTML5 parser, which gracefully handles malformed tags. For older versions, enable the built-in parser with `$crawler = new Crawler($html, 'https://example.com');`. Test edge cases early, especially with legacy HTML.
- What are the performance implications of DomCrawler for large-scale scraping?
- Memory usage scales with document size. For documents >10MB, consider streaming or chunking logic. DomCrawler isn’t optimized for distributed scraping—pair it with Symfony HttpClient and middleware for rate limiting or proxies if needed.
- Can I use DomCrawler for XML parsing in Laravel, or is it HTML-focused?
- DomCrawler supports XML but is optimized for HTML. For complex XML schemas, use XPath queries directly. Test with your specific XML structure, as nested or malformed XML may require custom logic.
- How does DomCrawler compare to Guzzle or Goutte for Laravel scraping?
- DomCrawler is lighter (~1MB) and focuses on DOM parsing, while Goutte (built on DomCrawler) adds HTTP client and JavaScript support. Use DomCrawler alone for static parsing; Goutte if you need HTTP interactions or form submissions.
- What Laravel versions and PHP requirements does DomCrawler support?
- Laravel 10+ requires `symfony/dom-crawler:^8.0` (PHP 8.4+). Laravel 9.x uses `^7.4` (PHP 8.1+). Avoid `^6.4` (PHP 7.4–8.0) unless maintaining legacy systems. Check Symfony’s [upgrade guide](https://symfony.com/doc/current/setup/upgrade.html) for breaking changes.
- How can I test DomCrawler in Laravel’s PHPUnit?
- Use Laravel’s `assertSelectorTextContains()` in HTTP tests or manually assert Crawler results. Example: `$response = $this->get('/page'); $crawler = new Crawler($response->getContent()); $this->assertEquals('Expected', $crawler->filter('h1')->text());` Mock dependencies for isolated testing.
Weaver
How can I help you explore Laravel packages today?