Bot Detect Laravel Package

Getting Started

Minimal Setup in Laravel

Install via Composer:

composer require vipx/bot-detect

Register the service provider in config/app.php:

'providers' => [
    // ...
    Vipx\BotDetect\BotDetectServiceProvider::class,
],

Publish the config (optional):

php artisan vendor:publish --provider="Vipx\BotDetect\BotDetectServiceProvider"

First Use Case: Middleware for Bot Detection

Create a middleware (app/Http/Middleware/DetectBots.php):

<?php

namespace App\Http\Middleware;

use Closure;
use Vipx\BotDetect\Facades\BotDetect;

class DetectBots
{
    public function handle($request, Closure $next)
    {
        $bot = BotDetect::detect($request->userAgent(), $request->ip());

        if ($bot) {
            // Log bot activity or block request
            return response('Bot detected', 403);
        }

        return $next($request);
    }
}

Register the middleware in app/Http/Kernel.php:

protected $middleware = [
    // ...
    \App\Http\Middleware\DetectBots::class,
];

Implementation Patterns

1. Service Container Integration

Leverage Laravel's service container for dependency injection:

use Vipx\BotDetect\BotDetector;

class SomeService
{
    protected $botDetector;

    public function __construct(BotDetector $botDetector)
    {
        $this->botDetector = $botDetector;
    }

    public function checkRequest($request)
    {
        $bot = $this->botDetector->detect(
            $request->userAgent(),
            $request->ip()
        );

        // Handle bot detection logic
    }
}

2. Facade Usage (Recommended for Simplicity)

Use the provided facade for cleaner code:

use Vipx\BotDetect\Facades\BotDetect;

class Controller
{
    public function index()
    {
        $bot = BotDetect::detect(
            request()->userAgent(),
            request()->ip()
        );

        if ($bot) {
            return response()->json(['message' => 'Bot detected']);
        }

        // Normal request handling
    }
}

3. Caching Strategy

Configure caching in config/bot-detect.php:

'cache' => [
    'enabled' => env('BOT_DETECT_CACHE_ENABLED', true),
    'driver' => env('BOT_DETECT_CACHE_DRIVER', 'file'),
    'path' => storage_path('framework/cache/bot-detect'),
],

4. Custom Bot Metadata

Extend the bot list by publishing and modifying the YAML files:

php artisan vendor:publish --tag="bot-detect-config"

Edit config/bot-detect/extended.yml to add custom bots.

5. Event-Based Detection

Listen for bot detection events:

use Vipx\BotDetect\Events\BotDetected;

class BotDetectionListener
{
    public function handle(BotDetected $event)
    {
        // Log bot activity
        \Log::info('Bot detected: '.$event->bot->name);

        // Block specific bots
        if ($event->bot->name === 'Googlebot') {
            abort(403, 'Googlebot access denied');
        }
    }
}

Register the listener in EventServiceProvider:

protected $listen = [
    \Vipx\BotDetect\Events\BotDetected::class => [
        \App\Listeners\BotDetectionListener::class,
    ],
];

Gotchas and Tips

Common Pitfalls

1. False Positives: Some legitimate user agents may match bot patterns. Test thoroughly with real traffic.
2. Performance Impact: Disable caching ('cache' => ['enabled' => false]) if you experience issues with large bot lists.
3. IP-Based Detection: Some bots may spoof user agents but reveal themselves via IP. Combine both detection methods.

Debugging Tips

1. Enable Debug Mode:

   $bot = BotDetect::detect($agent, $ip, ['debug' => true]);
   

   This will log detection attempts to storage/logs/bot-detect.log.

2. Check Cache Issues:

   • Ensure cache directory is writable (storage/framework/cache/bot-detect).
   • Clear cache when updating bot metadata:

     php artisan cache:clear
     

3. Update Bot Metadata:

   • Monitor the phpBB Manage_Bots project for updates.
   • Consider contributing new bot patterns to the package.

Configuration Quirks

1. Custom Metadata Loaders: The package supports YAML out-of-the-box. For other formats (XML, JSON), extend the MetadataLoader interface:

   class JsonFileLoader implements MetadataLoaderInterface
   {
       // Implement load() method
   }
   

   Register it in the service provider.

2. Case Sensitivity: Bot detection is case-insensitive by default. Override in config:

   'case_sensitive' => false,
   

3. Partial Matches: The detector uses regex patterns. For strict matching, adjust the match_strategy:

   'match_strategy' => 'exact', // 'regex' (default), 'exact', or 'contains'
   

Extension Points

1. Custom Bot Actions:

   event(new BotDetected($bot, $request))
       ->listen(function ($event) {
           // Custom logic for specific bots
           if ($event->bot->name === 'Bingbot') {
               // Special handling
           }
       });
   

2. Dynamic Bot Lists: Fetch updated bot lists from an external API and merge with the existing YAML:

   $externalBots = json_decode(file_get_contents('https://api.example.com/bots'), true);
   $mergedBots = array_merge($existingBots, $externalBots);
   

3. Rate Limiting for Bots: Combine with Laravel's rate limiting:

   $bot = BotDetect::detect($request->userAgent(), $request->ip());
   
   if ($bot) {
       $key = 'bot:'.$bot->name.':'.$request->ip();
       if (Cache::has($key)) {
           abort(429, 'Too many requests');
       }
       Cache::put($key, true, now()->addMinutes(1));
   }