Ai Amazee Ai Platform Laravel Package

Getting Started

Minimal Setup for Laravel/Symfony

1. Install the Package:

   composer require symfony/ai-amazeeai-platform
   

   For Laravel, ensure Symfony’s HttpClient is installed:

   composer require symfony/http-client
   

2. Configure LiteLLM Proxy: Add your amazee.ai API key and LiteLLM proxy endpoint to your config:

   // config/amazee_ai.php (Laravel) or config/packages/amazee_ai.yaml (Symfony)
   'api_key' => env('AMAZEE_AI_API_KEY'),
   'proxy_url' => 'https://proxy.litellm.ai/v1',
   'default_model' => 'gpt-3.5-turbo',
   'fallback_models' => ['mistral-7b', 'llama-2-70b'],
   

3. First Use Case: Basic Completion

   use Symfony\AI\Client;
   use Symfony\AI\Provider\AmazeeAiProvider;
   
   // Laravel Example
   $client = new Client(new AmazeeAiProvider(config('amazee_ai.api_key')));
   $response = $client->complete('Explain Laravel in 3 sentences');
   echo $response->getContent();
   

4. Verify Streaming Support (Symfony 7+):

   $stream = $client->streamComplete('Explain Laravel...');
   foreach ($stream as $delta) {
       echo $delta->getContent(); // Uses DeltaInterface
   }
   

Implementation Patterns

Core Workflows

1. Multi-Provider Routing

Leverage LiteLLM’s proxy to dynamically route requests:

// config/amazee_ai.php
'model_routing' => [
    'gpt-4' => ['primary' => true, 'fallback' => ['mistral-7b']],
    'gpt-3.5-turbo' => ['primary' => true, 'cost_threshold' => 0.002],
],

Trigger fallback logic via Symfony’s AiEvent (Symfony) or custom Laravel events.

2. Streaming Responses

Use DeltaInterface for type-safe streaming in Symfony:

$stream = $client->streamChat([
    'model' => 'gpt-3.5-turbo',
    'messages' => [['role' => 'user', 'content' => 'Hello']],
]);

foreach ($stream as $delta) {
    if ($delta instanceof \Symfony\AI\Message\ChatMessageDelta) {
        echo $delta->getContent();
    }
}

For Laravel, wrap the stream in a Generator or use Symfony\Component\StreamedResponse.

3. Fallback Logic

Implement a custom Provider to handle failures:

use Symfony\AI\Provider\ProviderInterface;

class CustomAmazeeAiProvider implements ProviderInterface
{
    public function complete(string $prompt, array $options = []): ResponseInterface
    {
        try {
            return $this->amazeeClient->complete($prompt, $options);
        } catch (Exception $e) {
            return $this->fallbackClient->complete($prompt, $options);
        }
    }
}

4. Integration with Laravel Services

Bind the client to Laravel’s container:

// app/Providers/AppServiceProvider.php
public function register()
{
    $this->app->singleton('amazee.ai', function ($app) {
        return new Client(new AmazeeAiProvider($app['config']['amazee_ai.api_key']));
    });
}

Inject via constructor:

public function __construct(private Client $amazeeClient) {}

Advanced Patterns

Dynamic Model Selection

Use Laravel’s config() or Symfony’s ParameterBag to switch models at runtime:

$model = config('amazee_ai.dynamic_model') ?? 'gpt-3.5-turbo';
$response = $client->complete('...', ['model' => $model]);

Observability

Log AI requests/responses with Laravel’s Log or Symfony’s Monolog:

$client->complete('...', ['model' => 'gpt-4'])
    ->then(function (ResponseInterface $response) {
        \Log::info('AI Response', ['content' => $response->getContent()]);
    });

Caching Responses

Cache frequent queries using Laravel’s Cache or Symfony’s Cache component:

$cacheKey = 'ai:'.md5($prompt);
$response = \Cache::remember($cacheKey, now()->addMinutes(5), function () use ($client, $prompt) {
    return $client->complete($prompt);
});

Gotchas and Tips

Pitfalls

1. Laravel-Symfony Integration Gaps:

   • Issue: Symfony’s AiEvent and ProviderInterface won’t work out-of-the-box in Laravel.
   • Fix: Use Symfony’s HttpClient directly or create a minimal Laravel adapter:

     // app/Services/AmazeeAiAdapter.php
     class AmazeeAiAdapter
     {
         public function __construct(private HttpClientInterface $client) {}
     
         public function complete(string $prompt): string
         {
             $response = $this->client->request('POST', 'https://proxy.litellm.ai/v1/chat/completions', [
                 'json' => ['model' => 'gpt-3.5-turbo', 'messages' => [['role' => 'user', 'content' => $prompt]]],
             ]);
             return json_decode($response->getContent(), true)['choices'][0]['message']['content'];
         }
     }
     

2. Streaming Quirks:

   • Issue: Laravel’s Symfony\Component\StreamedResponse may not handle DeltaInterface natively.
   • Fix: Convert deltas to JSON chunks manually:

     $stream = $client->streamComplete('...');
     return response()->stream(function () use ($stream) {
         foreach ($stream as $delta) {
             echo json_encode(['content' => $delta->getContent()])."\n";
         }
     });
     

3. API Key Management:

   • Issue: Hardcoding keys in config violates security best practices.
   • Fix: Use Laravel’s .env or Symfony’s ParameterBag with environment variables:

     'api_key' => env('LITELLM_API_KEY'), // Laravel
     // or
     '%env(AMAZEE_AI_API_KEY)%' // Symfony
     

4. Model Routing Complexity:

   • Issue: Custom routing logic may break if LiteLLM’s proxy endpoints change.
   • Fix: Test fallback models in staging and monitor with Laravel Scout or Symfony’s Profiler.

5. Rate Limiting:

   • Issue: LiteLLM’s free tier has strict limits (e.g., 10K requests/month).
   • Fix: Implement exponential backoff in Laravel:

     use Symfony\Component\Cache\Adapter\AdapterInterface;
     
     public function completeWithRetry(string $prompt, AdapterInterface $cache): string
     {
         $key = 'ai:rate_limit:'.md5($prompt);
         if ($cache->get($key, false)) {
             throw new \RuntimeException('Rate limit exceeded');
         }
         $cache->set($key, true, 60);
         return $this->client->complete($prompt);
     }
     

Debugging Tips

1. Enable Verbose Logging:

   // Laravel
   \Log::debug('AI Request', ['prompt' => $prompt, 'options' => $options]);
   // Symfony
   $this->logger->debug('AI Request', ['prompt' => $prompt]);
   

2. Inspect Raw Responses: Use dd() or var_dump() to debug LiteLLM’s proxy responses:

   $response = $client->complete('...');
   dd($response->getContent()); // Raw JSON
   

3. Validate API Keys: Test connectivity with a simple request:

   curl -X POST https://proxy.litellm.ai/v1/chat/completions \
     -H "Authorization: Bearer $API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Test"}]}'
   

Extension Points

1. Custom Providers: Extend AmazeeAiProvider to add pre/post-processing:

   class EnhancedAmazeeAiProvider extends AmazeeAiProvider
   {
       public function complete(string $prompt, array $options = []): ResponseInterface
       {
           $prompt = $this->preprocessPrompt($prompt);
           $response = parent::complete($prompt, $options);
           return $this->postprocessResponse($response);