Getting Started

Minimal Setup

Installation:

composer require cdaguerre/robots-bundle

Add to config/app.php under providers:

Dag\RobotsBundle\RobotsBundle::class,

Basic Configuration: Define rules in config/packages/dag_robots.yaml (or config/dag_robots.yaml if using Symfony Flex):
```
dag_robots:
  rules:
    - { route: 'homepage', tags: ['index', 'follow'] }  # Defaults to allow all bots
```
First Use Case:
- Route homepage will now include <meta name="robots" content="index, follow"> in its response.
- Verify by inspecting the <head> of your homepage in a browser or via curl -I.

Implementation Patterns

Core Workflow

Define Rules: Use YAML to map routes to robots meta tags (e.g., noindex, nofollow, noarchive). Example for dynamic exclusion:

dag_robots:
  rules:
    - { route: 'admin_*', tags: ['noindex', 'nofollow'] }  # Wildcard routes
    - { route: 'sitemap', tags: ['index', 'follow', 'max-snippet:-1'] }

Integration with Controllers: The bundle auto-injects headers via Symfony’s Response event system. No manual tagging required. Override per-request via a temporary rule:
```
// In a controller
$this->get('dag_robots.manager')->addTemporaryRule(
    $request->get('_route'),
    ['noindex', 'nofollow']
);
```
Host-Specific Rules: Restrict rules to specific domains (e.g., block search engines only for staging):
```
- { route: '*', tags: ['noindex'], hosts: ['staging.example.com'] }
```

Combining with Other Meta Tags: Use Symfony’s Twig to merge with existing meta tags:

{% block meta %}
    {{ parent() }}
    {% if robots_content is defined %}
        <meta name="robots" content="{{ robots_content }}">
    {% endif %}
{% endblock %}

Advanced Patterns

Dynamic Rules via Services: Inject RobotsManager into services to generate rules programmatically:

$manager->addRule(
    'user_profile',
    ['noindex', 'nofollow'],
    ['*.example.com']  // Regex-supported hosts
);

Conditional Rules: Use Symfony’s event_dispatcher to modify rules at runtime:

// src/EventListener/RobotsListener.php
public function onKernelRequest(GetResponseEvent $event) {
    if ($event->getRequest()->get('_route') === 'login') {
        $event->getRequest()->attributes->set('robots_tags', ['noindex']);
    }
}

Fallback Rules: Define a catch-all rule to ensure no route is accidentally exposed:

dag_robots:
  rules:
    - { route: '*', tags: ['noindex'] }  # Last rule applies to unmatched routes

Gotchas and Tips

Common Pitfalls

Rule Precedence:
- Rules are evaluated top-down; later rules override earlier ones for the same route.
- Fix: Order rules explicitly or use hosts to scope them.
Wildcard Routes:
- route: '*' matches all routes, but may conflict with explicit routes.
- Tip: Place wildcard rules last in the config.
Caching Issues:
- Headers are added via Symfony’s Response event, which may be cached by proxies (e.g., Varnish).
- Fix: Ensure Vary: Cookie or Cache-Control: no-cache headers are set for dynamic rules.
Twig Debugging:
- Meta tags may not appear in dev tools if Twig’s debug mode strips them.
- Tip: Check the raw HTTP response (curl -v) or disable Twig’s debug temporarily.
Host Matching:
- Hosts use exact matches (e.g., www.example.com ≠ example.com).
- Tip: Use regex for flexibility:
```
hosts: ['.*\.example\.com']  # Matches all subdomains
```

Debugging Tips

Log Rules: Enable debug mode in config/packages/dag_robots.yaml:

dag_robots:
  debug: true  # Logs applied rules to Symfony's logger

Inspect Headers: Use bin/console debug:config dag_robots to verify loaded rules. Check HTTP responses with:
```
curl -I http://your-site.com/homepage
```

Override in Tests: Reset rules in PHPUnit tests:

$this->get('dag_robots.manager')->resetRules();

Extension Points

Custom Tag Providers: Extend RobotsManager to fetch rules from a database:

// src/Service/RobotsDbProvider.php
class RobotsDbProvider implements RobotsProviderInterface {
    public function getRules() {
        return $this->db->findAll(); // Return array of rule configs
    }
}

Event Listeners: Modify rules dynamically via kernel.request events:

// src/EventListener/DynamicRobotsListener.php
public function onKernelRequest(GetResponseEvent $event) {
    $route = $event->getRequest()->get('_route');
    if (str_starts_with($route, 'api_')) {
        $event->getRequest()->attributes->set('robots_tags', ['noindex']);
    }
}

Alternative Output Formats: Override the RobotsTagGenerator to output X-Robots-Tag HTTP headers instead of meta tags:
```
// config/packages/dag_robots.yaml
dag_robots:
  output_format: http_header
```

Robots Bundle Laravel Package