Weave Code
Crawler Laravel Package
spatie/crawler
PHP web crawler that discovers links concurrently via Guzzle, with optional JavaScript rendering powered by Chrome/Puppeteer. Configure depth, internal-only rules, and callbacks for per-page handling, plus a fake mode to test crawl logic without real HTTP requests.
|
Package
|
Description
|
Stars
|
Likes
|
Forks
|
Downloads
|
Issues
|
Score
|
Opportunity
|
License
|
Last Release
|
|
|---|---|---|---|---|---|---|---|---|---|---|---|
| spatie/http-status-check | CLI tool to crawl a website and report HTTP status codes for every link. Scan internal and optionally external URLs, control concurrency for speed, and export failing (non-2xx/3xx) links to a file for quick auditing. | 599 | 601 | 87 | 91 | 0 | 6.1 | 0.8 | MIT | 2 years ago | |
| spatie/laravel-export | Export your Laravel app as a static site bundle. Crawls your routes to generate HTML for discovered URLs and includes the public directory for assets. Ideal for blogs/sites built with Laravel, then deployed to Netlify or any static host. | 669 | 670 | 63 | 5K | 0 | 32.6 | 15.5 | MIT | 1 week ago | |
| spatie/laravel-link-checker | Unmaintained: Artisan command to crawl your Laravel app and check internal/external links. Logs URLs that don’t return 2xx/3xx responses and can email broken-link reports. Configurable base URL, link-check profiles, reporters, and concurrency settings. | 261 | 258 | 42 | 99 | 0 | 4.9 | 1.4 | MIT | 3 years ago | |
| spatie/laravel-site-search | Crawl and index your Laravel site for fast full-text search—like a private Google. Highly customizable crawling and indexing, with concurrent requests. Uses SQLite FTS5 by default (no external services), or Meilisearch for advanced features. | 307 | 309 | 24 | 2K | 0 | 26.7 | 14.5 | MIT | 3 months ago | |
| spatie/laravel-sitemap | Generate XML sitemaps for Laravel by crawling your site or building them manually. Add extra URLs, set last-modified dates, and include models via a simple interface. Write sitemaps to disk with a fluent, developer-friendly API. | 2,598 | 2,629 | 299 | 613K | 0 | 47.7 | 37.3 | MIT | 2 months ago | |
| spatie/mixed-content-scanner | Scan a website for mixed content by crawling pages and flagging insecure http:// resources in common HTML tags (img, script, iframe, link, etc.). Use MixedContentScanner with a logger to report where mixed content is found or missing. | 104 | 105 | 18 | 445 | 0 | 10.1 | 9.9 | MIT | 3 years ago | |
| spatie/mixed-content-scanner-cli | CLI tool to detect mixed content on HTTPS sites. Crawls pages and reports HTML elements whose URLs use http:// (images, scripts, iframes, forms, etc.). Install via Composer globally and run: mixed-content-scanner scan . | 92 | 92 | 20 | 0 | 0 | 3.8 | — | MIT | 3 years ago | |
| spatie/one-package-to-rule-them-all | Pulls in all Spatie packages at once so you don’t have to require them individually. Install via Composer, optionally publish the config, and get the full Spatie “magic” in one go for your Laravel app. | 60 | 60 | 6 | 0 | 0 | 0.8 | — | MIT | 5 years ago |
Weaver
How can I help you explore Laravel packages today?