Weave Code
Crawler Laravel Package
spatie/crawler
A powerful PHP package for crawling websites, using Guzzle promises to fetch multiple URLs concurrently. It can render JavaScript via Chrome and Puppeteer, restrict crawls to internal links or depth, collect found URLs, and fake HTTP responses for testing crawl logic.
|
Package
|
Score
|
Description
|
Stars
|
Likes
|
Forks
|
Downloads
|
Issues
|
Score
|
Opportunity
|
License
|
Last Release
|
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| spatie/query-string | 0.93 | Manipulate URLs by building and updating query strings in PHP. Toggle parameters or values, add JSON:API-style filters and sorts, and manage pagination. Useful for generating links and keeping state in apps without manual query parsing. | 161 | 162 | 9 | 19K | 0 | 2.0 | 28.8 | MIT | 5 years ago | |
| spatie/file-system-watcher | 0.93 | React to file system changes from PHP using Spatie’s watcher powered by Node’s chokidar. Watch one or more paths and run callbacks on create, update, delete, or any change events. Useful for build tools, dev workflows, and background monitoring. | 251 | 251 | 25 | 1M | 0 | 6.0 | 50.2 | MIT | 11 months ago | |
| spatie/dns | 0.92 | Spatie's DNS package fetches DNS records for any domain via dig, supporting A, AAAA, CNAME, NS, PTR, SOA, MX, SRV, TXT, DNSKEY, CAA and NAPTR types. Filter by one or multiple record types and read record details through simple methods. | 613 | 621 | 98 | 192K | 0 | 35.1 | 40.4 | MIT | 1 week ago | |
| spatie/7to5 | 0.92 | Abandoned. Converts PHP 7.0 code to PHP 5, useful when developing on PHP 7 but deploying to PHP 5. Provides a CLI to convert entire directories and rewrites features like anonymous classes, type hints, return types, and null coalescing. | 177 | 175 | 32 | 46 | 0 | 2.6 | 1.0 | MIT | 8 years ago | |
| spatie/regex | 0.92 | Cleaner, safer wrapper around PHP’s preg_* functions. Provides Regex::match/matchAll/replace with MatchResult objects, easy access to groups, sensible defaults, and callback-based replacements—no by-reference variables or confusing false/null error handling. | 1,121 | 1,133 | 49 | 2M | 0 | 11.6 | 43.8 | MIT | 4 years ago | |
| spatie/image | 0.92 | spatie/image provides an expressive, chainable PHP API for common image manipulation. Resize to fit a bounding box without altering aspect ratio, apply greyscale, adjust brightness and quality, rotate, sharpen, and save or overwrite images in just a few lines. Ideal for Laravel and PHP apps. | 1,367 | 1,389 | 143 | 4M | 0 | 40.9 | 56.6 | MIT | 3 days ago | |
| spatie/test-time | 0.92 | Control time in tests with Spatie TestTime—freeze or advance Carbon-based timestamps dynamically for time-sensitive assertions. Simplify testing date-dependent logic by manually adjusting time without altering real-world clocks. Works seamlessly with Carbon’s methods like addYear(), subDay(), or cus... | 208 | 208 | 9 | 246K | 0 | 5.2 | 42.4 | MIT | 2 years ago | |
| spatie/url | 0.89 | Immutable URL parser/builder for PHP. Parse scheme, host, path and query, then fluently transform parts (withHost, withPath, withScheme) and manage query parameters. Supports allowed/sanitized schemes for safer URL handling. | 738 | 751 | 59 | 679K | 0 | 12.4 | 40.9 | MIT | 2 years ago | |
| spatie/visit | 0.89 | Human-friendly curl for the terminal. Visit any URL and display its response with colorized output (HTML via bat, JSON via jq), plus status code and response time. Supports custom HTTP methods and options like following redirects. | 390 | 390 | 14 | 2 | 0 | 7.5 | 0.0 | — | 1 year ago | |
| spatie/mixed-content-scanner | 0.89 | Scan a website for mixed content by crawling pages and flagging insecure http:// resources in common HTML tags (img, script, iframe, link, etc.). Use MixedContentScanner with a logger to report where mixed content is found or missing. | 105 | 106 | 18 | 58 | 0 | 10.2 | 2.6 | MIT | 3 years ago | |
| spatie/http-status-check | 0.88 | CLI tool to crawl a website and report HTTP status codes for every link. Scan internal and optionally external URLs, control concurrency for speed, and export failing (non-2xx/3xx) links to a file for quick auditing. | 600 | 603 | 85 | 32 | 0 | 6.1 | 0.2 | MIT | 3 years ago | |
| spatie/laravel-prometheus | 0.87 | Export key Laravel app metrics to Prometheus via an easy metric registry and a /prometheus endpoint. Ships with built-in queue and Horizon metrics, plus an optional security layer to keep sensitive data private. Visualize the results in Grafana. | 289 | 290 | 40 | 232K | 0 | 27.1 | 46.4 | MIT | 1 month ago | |
| spatie/invade | 0.86 | Spatie's invade function lets you read, write, and call private properties and methods of any PHP object, including static ones, through a simple proxy—handy for testing or working with legacy code. | 341 | 343 | 20 | 7M | 0 | 24.7 | 64.6 | MIT | 2 weeks ago | |
| spatie/laravel-site-search | 0.86 | Crawl and index your entire Laravel site for private full-text search, like a Google for your own content. Crawling and indexing are highly customizable, and it works out of the box with a local SQLite FTS5 database—no external dependencies—or optionally Meilisearch. | 310 | 312 | 25 | 2K | 0 | 23.4 | 12.5 | MIT | 4 months ago | |
| spatie/laravel-data | 0.86 | Spatie Laravel Data lets you define rich data objects once and reuse them as form requests, API transformers, and TypeScript definitions. It supports automatic validation, resource transformation, lazy properties, and constructing objects from request data. | 1,801 | 1,815 | 293 | 4M | 6 | 51.4 | 63.7 | MIT | 1 week ago | |
| vdb/php-spider | 0.86 | A configurable PHP web spider/crawler with a modular architecture for scraping and traversing sites. Define start URLs, URI filters, processors, and post-processors to control crawl depth, skip duplicate pages, and extract or analyze content with custom callbacks. | 1,340 | 1,357 | 228 | 2K | 1 | 14.0 | 5.8 | MIT | — | |
| codeguy/arachnid | 0.85 | Arachnid is a lightweight PHP web crawler and scraper with a simple, expressive API for fetching pages, following links, and extracting content. Use it to automate site navigation and data collection in Laravel or any PHP app, and chain requests and selectors to build custom spiders quickly. | 254 | — | 59 | — | 5 | 4.4 | 1.1 | MIT | 5 years ago |
|
| spatie/laravel-auto-discoverer | 0.85 | Quickly discover classes, interfaces, enums, and traits in your PHP or Laravel app that meet certain conditions, like finding all classes implementing an interface. Built-in caching keeps it fast in production. | 179 | 179 | 18 | 5 | 1 | 22.0 | 0.2 | MIT | 4 months ago | |
| mediamonks/crawler | 0.85 | A Laravel/PHP package for crawling websites: follow links, extract page content, and run custom handlers per URL. Useful for scraping, link checking, sitemap generation, and content audits, with configurable crawl limits and Laravel-friendly integration. | 21 | 21 | 4 | 5 | 0 | 0.2 | 1.2 | MIT | — | |
| vipnytt/sitemapparser | 0.85 | A lightweight PHP library for parsing XML sitemaps and sitemap indexes. It recursively follows sitemap index files, extracts URLs, handles gzip compression, detects sitemap types, and supports robots.txt discovery with no external dependencies. | 78 | — | 33 | — | 0 | 8.2 | — | MIT | 3 months ago |
|
Weaver
How can I help you explore Laravel packages today?