Weave Code
Robots Txt Laravel Package
spatie/robots-txt
Generate and serve a correct robots.txt in Laravel with an expressive API. Add user-agents, allow/disallow rules, sitemaps and host directives, then publish it via a route or controller—perfect for managing crawler access per environment.
|
Package
|
Description
|
Stars
|
Likes
|
Forks
|
Downloads
|
Issues
|
Score
|
Opportunity
|
License
|
Last Release
|
|
|---|---|---|---|---|---|---|---|---|---|---|---|
| atoolo/crawler-teaser-indexer | — | 0 | 0 | 0 | 10 | 0 | 20.7 | 16.9 | MIT | 2 weeks ago | |
| spatie/crawler | PHP web crawler with concurrent requests via Guzzle. Crawl internal/external links, limit depth, and collect discovered URLs. Supports JavaScript-rendered sites using Chrome/Puppeteer. Includes faking responses for testing crawl logic without real HTTP calls. | 2,828 | 2,835 | 367 | 733K | 0 | 55.0 | 38.8 | MIT | 1 month ago | |
| spatie/one-package-to-rule-them-all | Pulls in all Spatie packages at once so you don’t have to require them individually. Install via Composer, optionally publish the config, and get the full Spatie “magic” in one go for your Laravel app. | 60 | 60 | 6 | 0 | 0 | 0.8 | — | MIT | 5 years ago |
Weaver
How can I help you explore Laravel packages today?