- How do I integrate CrawlerDetect into Laravel middleware for request-level bot detection?
- Create a middleware class extending `Closure` or `HandleCrawlerRequests` (if using the Laravel wrapper). Inject the `CrawlerDetect` instance via the constructor, then call `$crawlerDetect->isCrawler()` in the `handle()` method. Bind the middleware in `app/Http/Kernel.php` under the `$routeMiddleware` array for global or route-specific use.
- Can I use CrawlerDetect to block specific bots (e.g., AhrefsBot) while allowing others (e.g., Googlebot)?
- Yes. Use the `$data` array in `Fixtures/Crawlers.php` to customize rules—add `!AhrefsBot` to block it or `Googlebot` to whitelist it. For Laravel, override the config in `config/crawler-detect.php` to define granular blacklists/whitelists per endpoint or globally.
- Does CrawlerDetect work with Laravel’s Blade templates for conditional content serving?
- Yes. Use the `@crawler` directive (if using the Laravel wrapper) or manually inject the `CrawlerDetect` instance into your Blade views. Example: `@if(!app('crawler-detect')->isCrawler()) Show content to humans @endif`. For dynamic checks, pass the User-Agent string directly to `isCrawler()`.
- What Laravel versions and PHP versions does jaybizzle/crawler-detect support?
- The package supports PHP 7.1–8.4 and works with Laravel 5.5+. For Laravel 10+, ensure your `composer.json` dependencies align with the latest `jaybizzle/crawler-detect` (v1.3.x) and its PHP requirements. The core package is framework-agnostic but tested rigorously with modern Laravel versions.
- How can I test CrawlerDetect in my Laravel application before deploying to production?
- Use the `isCrawler()` method with a hardcoded User-Agent string (e.g., `$crawlerDetect->isCrawler('Bingbot')`) in your tests. For unit testing, mock the `Request` object to simulate bot traffic. The package’s 95%+ test coverage ensures reliability, but validate edge cases like spoofed headers in your CI pipeline.
- Is CrawlerDetect suitable for high-traffic Laravel applications? What’s the performance impact?
- Yes. Benchmarks show <1ms detection latency even with 1,000+ crawler patterns. For high-traffic apps, cache detection results in Laravel’s cache driver (e.g., Redis) or preload the `CrawlerDetect` instance as a singleton in `AppServiceProvider`. Avoid repeated regex checks for known bots like Googlebot.
- How do I extend CrawlerDetect to detect custom bots (e.g., internal monitoring tools) not in the default list?
- Edit the `$data` array in `Fixtures/Crawlers.php` or override the config in `config/crawler-detect.php`. Add a regex pattern for your bot’s User-Agent (e.g., `InternalMonitoringBot` with its signature) and submit a PR to the repo to contribute back. Test your additions with `tests/crawlers.txt` before deployment.
- Can I combine CrawlerDetect with Laravel’s fail2ban or IP blocking middleware for auto-ban?
- Yes. Use CrawlerDetect to identify bots, then integrate with Laravel’s `fail2ban` or middleware like `BlockThrottledRequests` to auto-ban persistent offenders. Log bot names via `getMatches()` and trigger IP blocks for suspicious patterns (e.g., `HTTP_FROM` spoofing). Pair with IP reputation checks for robustness.
- What are the alternatives to CrawlerDetect for Laravel, and when should I choose them?
- Alternatives include `spatie/bot-detector` (simpler, fewer patterns) or `league/uri` + custom regex (more control). Choose CrawlerDetect for its 1,000+ bot signatures, Laravel middleware support, and community-driven updates. Use `spatie/bot-detector` for lightweight needs or `league/uri` if you need fine-grained regex logic without a pre-built list.
- How do I handle false positives (e.g., legitimate users misclassified as bots) in production?
- Customize the `$data` array in `Fixtures/Crawlers.php` to refine patterns or use Laravel’s config to whitelist specific User-Agents. For edge cases, combine with JavaScript challenges (e.g., Cloudflare Bot Management) or IP-based allowlists. Monitor false positives via Laravel Telescope or Sentry to iteratively improve accuracy.