Ai Ollama Platform Laravel Package

Getting Started

Minimal Setup for Laravel

1. Install the Package:

   composer require symfony/ai-ollama-platform
   

   Note: Requires PHP 8.1+ and Symfony’s HTTP Client (install via symfony/http-client if missing).

2. Basic Chat Example:

   use Symfony\Component\AI\Ollama\OllamaClient;
   use Symfony\Component\HttpClient\HttpClient;
   
   $client = new OllamaClient(HttpClient::create());
   $response = $client->chat('llama3', 'Explain Laravel to a 5-year-old');
   echo $response->getContent();
   

3. Laravel Integration (HTTP Facade):

   use Illuminate\Support\Facades\Http;
   use Symfony\Component\AI\Ollama\OllamaClient;
   
   // Register a macro to adapt Laravel's HTTP to Symfony's client
   Http::macro('createClient', fn() => HttpClient::create());
   
   $ollama = new OllamaClient(Http::createClient());
   $result = $ollama->chat('mistral', 'Write a poem about Laravel');
   

4. First Use Case:

   • Chatbot: Stream responses for a real-time UI.

     $stream = $ollama->chatStream('llama3', 'Tell me a joke');
     foreach ($stream as $chunk) {
         echo $chunk->getContent(); // NDJSON chunk
     }
     

   • Embeddings: Generate vectors for search.

     $embedding = $ollama->embed('nomic-embed-text', 'Laravel is awesome');
     

Where to Look First

• API Reference: Ollama Docs (understand chat, embed, and streaming).
• Symfony AI Docs: Symfony AI for abstractions like Provider.
• Laravel Adapter: Focus on OllamaClient and HttpClient integration—avoid Symfony-specific components (e.g., Messenger) unless needed.

Implementation Patterns

Core Workflows

1. Chat Interactions (Synchronous/Streaming)

• Synchronous:

  $response = $ollama->chat('llama3', 'Summarize this: ' . $longText);
  $content = $response->getContent(); // JSON string
  $data = json_decode($content, true);
  

• Streaming (NDJSON):

  $stream = $ollama->chatStream('llama3', 'Explain Laravel');
  foreach ($stream as $chunk) {
      $delta = $chunk->getDelta(); // DeltaInterface
      echo $delta->getContent();   // Incremental text
  }
  

• Laravel Streaming: Use Laravel Echo/Pusher to broadcast chunks:

  $stream->onEach(function ($chunk) {
      broadcast(new OllamaChunk($chunk->getDelta()->getContent()));
  });
  

2. Embeddings for Search

$embedding = $ollama->embed('nomic-embed-text', 'Laravel vs Symfony');
$vector = $embedding->getEmbedding(); // Normalized array
// Store in vector DB (e.g., Qdrant)

3. Structured Outputs (v0.7.0+)

$response = $ollama->chat(
    'llama3',
    'Extract {name, age} from: "John Doe, 30"',
    ['structured_output' => true]
);
$data = json_decode($response->getContent(), true);
// $data = ['name' => 'John Doe', 'age' => 30]

4. Model Routing (Provider Abstraction)

• Define a Laravel service to route models dynamically:

  class OllamaModelProvider implements ProviderInterface {
      public function getModel(string $name): string {
          return match ($name) {
              'poetry' => 'mistral',
              'math' => 'llama3',
              default => $name,
          };
      }
  }
  

• Inject into OllamaClient:

  $client = new OllamaClient(HttpClient::create(), new OllamaModelProvider());
  

5. Audio Capabilities (Gemma Models)

$response = $ollama->chat('gemma:2b', 'Transcribe this audio: [base64]');

Integration Tips

• Laravel Service Container: Bind OllamaClient as a singleton:

  $app->singleton(OllamaClient::class, fn($app) =>
      new OllamaClient($app->make(HttpClient::class))
  );
  

• Error Handling: Wrap calls in try-catch for OllamaException:

  try {
      $response = $ollama->chat('invalid-model', '...');
  } catch (OllamaException $e) {
      Log::error('Ollama failed: ' . $e->getMessage());
  }
  

• Configuration: Use Laravel’s config to manage Ollama endpoints:

  // config/ollama.php
  return [
      'endpoint' => env('OLLAMA_ENDPOINT', 'http://localhost:11434'),
  ];
  

  Then inject into OllamaClient:

  $client = new OllamaClient(
      HttpClient::create(['base_uri' => config('ollama.endpoint')])
  );
  

Gotchas and Tips

Pitfalls

1. Symfony Abstraction Leakage:

   • Avoid using Symfony-specific components (e.g., Messenger, AI traits) in Laravel. Stick to OllamaClient and HttpClient.
   • Fix: Create Laravel interfaces for Symfony abstractions (e.g., ProviderInterface).

2. Streaming in Laravel:

   • NDJSON streaming may block Laravel’s synchronous request lifecycle. Use:
     • Queues: Dispatch streaming to a background job.
     • Broadcasting: Stream chunks via Laravel Echo/Pusher.
     • Example:

       $stream->onEach(function ($chunk) {
           OllamaChunkJob::dispatch($chunk->getDelta()->getContent());
       });
       

3. Model Catalog Management:

   • The package uses OllamaApiCatalog to list models. Cache results in Laravel’s cache or database to avoid repeated API calls:

     $catalog = $ollama->getCatalog();
     Cache::put('ollama_models', $catalog, now()->addHours(1));
     

4. Audio/Data Payloads:

   • Ollama expects base64-encoded audio/data. Encode files before sending:

     $audio = base64_encode(file_get_contents('audio.wav'));
     $response = $ollama->chat('gemma:2b', "Transcribe: $audio");
     

5. PHP Version Conflicts:

   • The package targets PHP 8.1+. If using Laravel <9, ensure your composer.json enforces:

     "config": {
         "platform": {
             "php": "8.1"
         }
     }
     

Debugging Tips

1. Enable HTTP Logging:

   $client = new OllamaClient(
       HttpClient::create(['headers' => ['Accept' => 'application/json']])
   );
   // Add middleware for debugging
   $client->getHttpClient()->addSubscriber(new \Symfony\Component\HttpClient\EventListener\DebugListener());
   

2. Validate NDJSON Streaming:

   • Use a tool like NDJSON Validator to check streaming responses. Example malformed chunk:

     {"content":"Hello"}  // Missing newline between chunks
     

   • Fix: Ensure chunks are newline-delimited (\n).

3. Ollama Server Issues:

   • Verify Ollama is running:

     curl http://localhost:11434/api/tags
     

   • Check logs for errors:

     journalctl -u ollama -f
     

4. Structured Output Parsing:

   • Validate JSON responses from structured_output:

     $response = $ollama->chat('llama3', '...', ['structured_output' => true]);
     $data = json_decode($response->getContent(), true);
     if (json_last_error() !== JSON_ERROR_NONE) {
         throw new \RuntimeException('Invalid structured output');
     }
     

Extension Points

1. Custom Providers:

   • Extend ProviderInterface for dynamic model routing:

     class CustomProvider implements ProviderInterface {
         public function getModel(string $name): string {
             // Logic to map user-friendly names to Ollama models
         }
     }
     

2. Middleware for Requests:

   • Add headers