Ai Ollama Platform Laravel Package

Getting Started

Minimal Setup for Laravel

1. Install the Package:

   composer require symfony/ai-ollama-platform
   

   Note: Requires PHP 8.1+ and Symfony’s HTTP Client (install via symfony/http-client if missing).

2. Basic Chat Example:

   use Symfony\Component\AI\Ollama\OllamaClient;
   use Symfony\Component\HttpClient\HttpClient;
   
   $client = new OllamaClient(HttpClient::create());
   $response = $client->chat('llama3', 'Explain Laravel to a 5-year-old');
   echo $response->getContent();
   

3. Laravel Integration (HTTP Facade):

   use Illuminate\Support\Facades\Http;
   use Symfony\Component\AI\Ollama\OllamaClient;
   
   // Register a macro to adapt Laravel's HTTP to Symfony's client
   Http::macro('createClient', fn() => HttpClient::create());
   
   $ollama = new OllamaClient(Http::createClient());
   $result = $ollama->chat('mistral', 'Write a poem about Laravel');
   

4. First Use Case:

   • Chatbot: Stream responses for a real-time UI.

     $stream = $ollama->chatStream('llama3', 'Tell me a joke');
     foreach ($stream as $chunk) {
         echo $chunk->getContent(); // NDJSON chunk
     }
     

   • Embeddings: Generate vectors for search.

     $embedding = $ollama->embed('nomic-embed-text', 'Laravel is awesome');
     

   • Cost tracking (v0.14.0+): Read token usage straight off the response and attribute it to a model.

     $response = $ollama->chat('llama3', 'Summarize this ticket');
     $usage = $response->getTokenUsage();
     
     $usage->getModel();          // NEW in 0.14: "llama3" — the model that produced these tokens
     $usage->getPromptTokens();
     $usage->getCompletionTokens();
     

Where to Look First

• API Reference: Ollama Docs (understand chat, embed, and streaming).
• Symfony AI Docs: Symfony AI for abstractions like Provider.
• Token usage / pricing: TokenUsage — as of v0.14.0 it carries the originating model name, which is what you need to price a request. This is the first place to look when building billing or quota features.
• Laravel Adapter: Focus on OllamaClient and HttpClient integration—avoid Symfony-specific components (e.g., Messenger) unless needed.

Implementation Patterns

Core Workflows

1. Chat Interactions (Synchronous/Streaming)

• Synchronous:

  $response = $ollama->chat('llama3', 'Summarize this: ' . $longText);
  $content = $response->getContent(); // JSON string
  $data = json_decode($content, true);
  

• Streaming (NDJSON):

  $stream = $ollama->chatStream('llama3', 'Explain Laravel');
  foreach ($stream as $chunk) {
      $delta = $chunk->getDelta(); // DeltaInterface
      echo $delta->getContent();   // Incremental text
  }
  

• Laravel Streaming: Use Laravel Echo/Pusher to broadcast chunks:

  $stream->onEach(function ($chunk) {
      broadcast(new OllamaChunk($chunk->getDelta()->getContent()));
  });
  

2. Embeddings for Search

$embedding = $ollama->embed('nomic-embed-text', 'Laravel vs Symfony');
$vector = $embedding->getEmbedding(); // Normalized array
// Store in vector DB (e.g., Qdrant)

3. Structured Outputs (v0.7.0+)

$response = $ollama->chat(
    'llama3',
    'Extract {name, age} from: "John Doe, 30"',
    ['structured_output' => true]
);
$data = json_decode($response->getContent(), true);
// $data = ['name' => 'John Doe', 'age' => 30]

4. Model Routing (Provider Abstraction)

• Define a Laravel service to route models dynamically:

  class OllamaModelProvider implements ProviderInterface {
      public function getModel(string $name): string {
          return match ($name) {
              'poetry' => 'mistral',
              'math' => 'llama3',
              default => $name,
          };
      }
  }
  

• Inject into OllamaClient:

  $client = new OllamaClient(HttpClient::create(), new OllamaModelProvider());
  

5. Cost Calculation with TokenUsage::getModel() (v0.14.0+)

The new model property on TokenUsage means you no longer have to thread the requested model name through your own code to price a call — the usage object tells you which model it belongs to. This matters most when a Provider remaps a friendly alias (poetry) to a real model (mistral): you now price the actual model.

• Price table via config:

  // config/ollama.php
  return [
      'endpoint' => env('OLLAMA_ENDPOINT', 'http://localhost:11434'),
      'pricing' => [ // per 1K tokens, in cents
          'llama3'            => ['prompt' => 0.0,  'completion' => 0.0],
          'mistral'           => ['prompt' => 0.02, 'completion' => 0.06],
          'nomic-embed-text'  => ['prompt' => 0.01, 'completion' => 0.0],
      ],
  ];
  

• A small calculator service:

  class OllamaCostCalculator
  {
      public function cost(TokenUsage $usage): float
      {
          $model  = $usage->getModel() ?? 'unknown';
          $prices = config("ollama.pricing.{$model}", ['prompt' => 0, 'completion' => 0]);
  
          return ($usage->getPromptTokens() / 1000) * $prices['prompt']
               + ($usage->getCompletionTokens() / 1000) * $prices['completion'];
      }
  }
  

• Persist usage per request (works well as a queued listener or a middleware-level observer):

  AiUsage::create([
      'user_id'           => $user->id,
      'model'             => $usage->getModel(),   // 0.14.0
      'prompt_tokens'     => $usage->getPromptTokens(),
      'completion_tokens' => $usage->getCompletionTokens(),
      'cost_cents'        => app(OllamaCostCalculator::class)->cost($usage),
  ]);
  

• Enforce quotas before dispatching expensive work:

  if ($user->ai_spend_cents >= $user->ai_budget_cents) {
      abort(429, 'AI budget exhausted');
  }
  

6. Audio Capabilities (Gemma Models)

$response = $ollama->chat('gemma:2b', 'Transcribe this audio: [base64]');

Integration Tips

• Laravel Service Container: Bind OllamaClient as a singleton:

  $app->singleton(OllamaClient::class, fn($app) =>
      new OllamaClient($app->make(HttpClient::class))
  );
  

• Error Handling: Wrap calls in try-catch for OllamaException:

  try {
      $response = $ollama->chat('invalid-model', '...');
  } catch (OllamaException $e) {
      Log::error('Ollama failed: ' . $e->getMessage());
  }
  

• Configuration: Use Laravel’s config to manage Ollama endpoints:

  // config/ollama.php
  return [
      'endpoint' => env('OLLAMA_ENDPOINT', 'http://localhost:11434'),
  ];
  

  Then inject into OllamaClient:

  $client = new OllamaClient(
      HttpClient::create(['base_uri' => config('ollama.endpoint')])
  );
  

• Centralise usage logging: Rather than calling getTokenUsage() at every call site, wrap OllamaClient in a thin decorator that records usage + cost once. When you later add a second provider, the decorator already handles multi-model accounting.
• Aggregate, don’t recompute: store raw model + token counts, and derive cost at query time if your prices change. Storing only cost_cents means a price correction requires a backfill.

Gotchas and Tips

Pitfalls

1. Symfony Abstraction Leakage:

   • Avoid using Symfony-specific components (e.g., Messenger, AI traits) in Laravel. Stick to OllamaClient and HttpClient.
   • Fix: Create Laravel interfaces for Symfony abstractions (e.g., ProviderInterface).

2. Streaming in Laravel:

   • NDJSON streaming may block Laravel’s synchronous request lifecycle. Use:
     • Queues: Dispatch streaming to a background job.
     • Broadcasting: Stream chunks via Laravel Echo/Pusher.
     • Example:

       $stream->onEach(function ($chunk) {
           OllamaChunkJob::dispatch($chunk->getDelta()->getContent());
       });
       

3. Model Catalog Management:

   • The package uses OllamaApiCatalog to list models. Cache results in Laravel’s cache or database to avoid repeated API calls:

     $catalog = $ollama->getCatalog();
     Cache::put('ollama_models', $catalog, now()->addHours(1));
     

4. Audio/Data Payloads:

   • Ollama expects base64-encoded audio/data. Encode files before sending:

     $audio = base64_encode(file_get_contents('audio.wav'));
     $response = $ollama->chat('gemma:2b', "Transcribe: $audio");
     

5. PHP Version Conflicts:

   • The package targets PHP 8.1+. If using Laravel <9, ensure your composer.json enforces:

     "config": {
         "platform": {
             "php": "8.1"
         }
     }
     

6. TokenUsage::getModel() is new in 0.14.0 — and not guaranteed non-null:

   • If you are running 0.13.x in production, getModel() does not exist. Pin your constraint (^0.14) before shipping cost-tracking code, otherwise you'll hit a BadMethodCallException/undefined-method error only on the billing path.
   • Treat the value as nullable and always have a fallback (?? $requestedModel) so a missing/blank model doesn't produce $0.00 costs silently.
   • Watch for model name drift: Ollama tags can include suffixes (llama3:8b, llama3:latest). Normalise (strip the tag) before looking up a price, or your price table lookup will miss and fall through to your zero-cost default.
   • Local Ollama models are effectively free to run, so cost is really notional. Decide up front whether you're tracking cost (budgeting against a paid provider later) or tracking tokens (rate limiting). Mixing the two leads to confusing dashboards.

Debugging Tips

1. Enable HTTP Logging:

   $client = new OllamaClient(
       HttpClient::create(['headers' => ['Accept' => 'application/json']])
   );
   // Add middleware for debugging
   $client->getHttpClient()->addSubscriber(new \Symfony\Component\HttpClient\EventListener\DebugListener());
   

2. Validate NDJSON Streaming:

   • Use a tool like NDJSON Validator to check streaming responses. Example malformed chunk:

     {"content":"Hello"}  // Missing newline between chunks
     

   • Fix: Ensure chunks are newline-delimited (\n).

3. Ollama Server Issues:

   • Verify Ollama is running:

     curl http://localhost:11434/api/tags
     

   • Check logs for errors:

     journalctl -u ollama -f
     

4. Structured Output Parsing:

   • Validate JSON responses from structured_output:

     $response = $ollama->chat('llama3', '...', ['structured_output' => true]);
     $data = json_decode($response->getContent(), true);
     if (json_last_error() !== JSON_ERROR_NONE) {
         throw new \RuntimeException('Invalid structured output');
     }
     

5. Auditing token accounting:

   • Log the raw TokenUsage (model, prompt, completion) alongside your computed cost for the first few weeks. If your cost column and the raw tokens disagree with expectations, the bug is almost always in the price table key, not the SDK.
   • Sum prompt + completion in a tinker session over a day's rows and compare to your quota enforcement counter — drift means a call path is bypassing your decorator.

Extension Points

1. Custom Providers:
   • Extend ProviderInterface for dynamic model routing:

     class CustomProvider implements ProviderInterface {
         public function getModel(string $name): string {
             // Logic to map user-friendly names to Ollama models
         }
     }
     

2. Usage Decorator:
   • Wrap OllamaClient to capture TokenUsage (including the new model property) uniformly — the cleanest place to add cost tracking, quotas, and per-tenant metrics without touching call sites.
3. Middleware for Requests:
   • Add headers