Ai Gateway Bundle Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require ai-gateway/ai-gateway-bundle
   

   Symfony Flex auto-registers the bundle; otherwise, add it to config/bundles.php.

2. Database Setup:

   php bin/console doctrine:database:create --if-not-exists
   php bin/console doctrine:schema:update --force
   

3. Configure Routes: Add to config/routes.yaml:

   ai_gateway:
       resource: .
       type: ai_gateway
   

4. First API Key: Use the CLI to create a provider, model, and key:

   php bin/console provider:create --name openai --format openai --api-key 'your-api-key'
   php bin/console model:create --alias test_model --provider openai --model 'gpt-3.5-turbo' --pricing-input 0.001 --pricing-output 0.002
   php bin/console key:create --team default --name 'dev-key' --key 'aigw_dev_key'
   

5. First Request: Inject GatewayInterface and call:

   $response = $this->gateway->chat(new NormalizedRequest(
       model: 'test_model',
       key: 'aigw_dev_key',
       messages: [['role' => 'user', 'content' => 'Hello']]
   ));
   

Implementation Patterns

Core Workflow

1. Provider Setup: Define providers (e.g., OpenAI, Anthropic) via CLI or dashboard. Each provider maps to an LLM endpoint.

   php bin/console provider:create --name mistral --format openai --base-url 'https://api.mistral.ai/v1'
   

2. Model Configuration: Create model aliases for provider-specific models with pricing:

   php bin/console model:create --alias mistral-7b --provider mistral --model 'mistral-7b' --pricing-input 0.001 --pricing-output 0.003
   

3. Model Chains: Group models into fallback chains (e.g., primary model → backup model):

   php bin/console chain:create --alias 'mistral-chain'
   php bin/console chain:step:add --chain mistral-chain --model mistral-7b --priority 1 --weight 80
   php bin/console chain:step:add --chain mistral-chain --model gpt-3.5-turbo --priority 2 --weight 100
   

4. Key Management: Assign API keys to teams with budget/rate limits:

   php bin/console key:create --team dev-team --name 'dev-key' --key 'aigw_dev_key' --budget-per-day 10
   

5. Usage in Code: Use GatewayInterface for unified LLM calls:

   $response = $this->gateway->chat(new NormalizedRequest(
       model: 'mistral-chain', // Uses fallback chain
       key: 'aigw_dev_key',
       messages: [...]
   ));
   

Integration Tips

1. Dependency Injection: Bind GatewayInterface to a specific chain or model dynamically:

   $this->gateway->setModel('mistral-7b'); // Override for a request
   

2. Streaming Responses: Use SSE for chat completions:

   $response = $this->gateway->chat($request, ['stream' => true]);
   

3. Cost Tracking: Log costs per request via ai_gateway_cost_dollars_total metric.

4. Caching: Enable deterministic SHA-256 caching for repeated prompts:

   ai_gateway:
       cache:
           enabled: true
   

5. Dashboard Integration: Embed the dashboard in admin panels or use token auth for restricted access.

Gotchas and Tips

Pitfalls

1. Key Validation:

   • Keys must match the aigw_* prefix (e.g., aigw_dev_key). Use hash('sha256', $rawKey) for CLI-generated keys.
   • Fix: Validate keys in middleware or use KeyStoreInterface to check hashes.

2. Model Chain Priorities:

   • Lower priority numbers execute first. Ensure weights sum to 100% per priority.
   • Fix: Use chain:list to debug chain steps.

3. Rate Limiting:

   • Default limits apply per key. Exceeding limits returns 429 Too Many Requests.
   • Fix: Adjust limits in the dashboard or via KeyRules.

4. Dashboard Token:

   • If tokenRequired: true, the dashboard redirects to a login form. Ensure the token is passed in all links.
   • Fix: Use ?token=YOUR_TOKEN in URLs or configure Symfony firewall.

5. Provider Timeouts:

   • Underlying LLM providers may timeout. Use retry config:

   ai_gateway:
       retry:
           max_attempts: 3
           delay: 1000
   

Debugging Tips

1. Request Logs: Query gateway_request_log for failed requests:

   SELECT * FROM gateway_request_log WHERE status_code != 200 ORDER BY created_at DESC;
   

2. CLI Debugging: Use provider:list and model:list to verify configurations:

   php bin/console provider:list
   php bin/console model:list
   

3. Metrics: Check Prometheus metrics for cost/latency:

   • ai_gateway_requests_total
   • ai_gateway_cost_dollars_total

4. Dashboard Filters: Use the dashboard’s "Failed Requests" tab to filter by status code or model.

Extension Points

1. Custom Providers: Extend AIGateway\Core\Provider\ProviderInterface for unsupported formats (e.g., gemini).

2. Middleware: Add pre/post-processing via AIGateway\Core\Middleware\MiddlewareStack:

   $stack->add(new class implements MiddlewareInterface {
       public function handle(Request $request, callable $next): Response {
           // Modify request/response
           return $next($request);
       }
   });
   

3. Event Listeners: Subscribe to ai_gateway.request events for logging/auditing:

   $dispatcher->addListener('ai_gateway.request', function (RequestEvent $event) {
       // Log request details
   });
   

4. Custom Dashboard Pages: Extend the dashboard by overriding Twig templates in templates/ai_gateway/.

5. CLI Commands: Add custom commands by implementing AIGateway\Command\CommandInterface and registering them in services.yaml.