Ai Ollama Platform Laravel Package

Technical Evaluation

Architecture Fit

• Laravel Compatibility: The package remains Symfony-first, requiring adaptation for Laravel’s ecosystem. Key Symfony dependencies (e.g., HttpClient, Messenger, AI traits) must be abstracted or replaced with Laravel equivalents (e.g., Http facade, Queues, custom interfaces). The Provider abstraction (v0.8.0) and model routing are valuable but need Laravel-specific implementations (e.g., service containers, interfaces). v0.14.0 does not change this posture — it is an additive, non-breaking minor release.
• Ollama API Abstraction: Provides a clean, standardized interface for Ollama’s HTTP API, reducing boilerplate for chat, embeddings, and streaming. Aligns well with Laravel’s service-oriented architecture if properly encapsulated. The DeltaInterface for structured outputs (v0.7.0) and audio capabilities (e.g., gemma:2b) are unique selling points for Laravel apps requiring multimodal AI.
• Cost/Usage Telemetry (New in v0.14.0): TokenUsage now carries a model property, enabling per-model token accounting and cost calculation. This is directly actionable in Laravel: persist TokenUsage rows to a ai_usage table, aggregate via Eloquent, and expose cost dashboards (Nova/Filament) or per-tenant budgets. It also closes the loop on the previously open ROI / self-host vs. cloud question by letting us normalize cost per model across providers behind the Provider abstraction.
• Streaming Support: NDJSON streaming (fixed in v0.7.0) is critical for real-time use cases (e.g., chatbots) but may require Laravel-specific event handling (e.g., broadcasting with Laravel Echo, Pusher, or custom event listeners). Laravel’s synchronous request lifecycle may need adjustments (e.g., using Swoole, ReactPHP, or queues for async streaming). Note: usage/cost metadata typically arrives at stream completion — plan for a final-usage callback or post-stream persistence step so v0.14.0 cost data is captured for streamed requests too.
• Extensibility: The Provider abstraction allows swapping Ollama for other backends, but Laravel would need custom adapters for non-Ollama providers (e.g., local LLMs, cloud APIs). The OllamaApiCatalog (v0.7.0) simplifies model management but may need caching in Laravel’s database/Redis. With v0.14.0, cost attribution keys off the same model identifiers, so catalog/model-name normalization should be treated as a first-class concern.

Integration Feasibility

• Low-Coupling Potential: Can be integrated as a composable service in Laravel, but Symfony-specific components (e.g., Messenger, AI traits) would need replacement or abstraction. The package’s dependency on Symfony’s HttpClient remains the biggest hurdle; Laravel’s Http facade or Guzzle can substitute with minimal effort.
• Ollama Dependency: Requires Ollama server infrastructure (Docker, Kubernetes, or managed service), adding operational overhead. Model management (pulling, updating, scaling) must be handled separately, ideally via Laravel tasks (e.g., scheduled commands) or Docker Compose.
• PHP/Laravel Version: Targets PHP 8.1+, which may require Laravel 9+ or manual upgrades. Laravel’s event loop (e.g., for streaming) may need adjustments (e.g., Swoole, ReactPHP, or Laravel Queues for async processing).
• Upgrade Feasibility (0.13 → 0.14): Treated as a drop-in minor upgrade. The TokenUsage::model addition is additive, but we must verify: (a) our HTTP-wrapper integration path does not depend on TokenUsage serialization shape, and (b) any custom DTOs mapping usage data are updated to include the new field. Estimated effort: < 1 day, plus test-catalogue refresh.
• Key Laravel Gaps:
  • No native symfony/ai support → Custom facades/services required.
  • Symfony’s Messenger → Laravel Queues/Horizon for async tasks.
  • Symfony’s HttpClient → Laravel’s Http facade or Guzzle.
  • Symfony’s DeltaInterface → Custom Laravel event listeners for streaming.
  • Cost/usage persistence → No Laravel-side ledger exists; we must build it (migration, model, aggregation job) to exploit v0.14.0.

Technical Risk

Key Questions

1. Symfony vs. Laravel Trade-offs:
   • Should we abstract Symfony’s AI component in Laravel (high effort) or use this package only for HTTP calls (lower effort)?
   • Would a Laravel-native Ollama client (e.g., olliwood/ollama-php) be simpler than this Symfony bridge?
2. Operational Complexity:
   • How will we manage Ollama models (updates, storage, scaling) in production? Options:
     • Docker Compose for local/dev.
     • Kubernetes for scaling.
     • Laravel scheduled commands to pull/update models.
   • What’s the failure mode if Ollama’s HTTP API is unavailable (e.g., retries, fallbacks, circuit breakers)?
3. Performance:
   • How will streaming (NDJSON) interact with Laravel’s synchronous request lifecycle (e.g., middleware, queues)?
     • Use Swoole or ReactPHP for async streaming.
     • Offload streaming to Laravel Queues with a worker.
   • Are there memory/CPU constraints when processing large responses (e.g., long conversations)?
     • Implement chunked processing or stream-to-disk for large outputs.
4. Cost & Usage Attribution (New):
   • What is our cost ledger schema for TokenUsage (tenant, user, feature, model, prompt/completion tokens, computed cost, timestamp)? Do we persist per-request or aggregate?
   • Which price table backs the calculation, and who owns updating it when models/prices change? Self-hosted Ollama has infra cost rather than per-token cost — do we model a notional per-token rate to enable apples-to-apples comparison with cloud APIs?
   • Do we need budget enforcement (per-tenant caps, alerts) built on top of this data, and should that gate requests pre-flight?
   • Is usage data captured for streamed responses, which are the bulk of chat traffic?
5. Long-Term Viability:
   • Is Symfony’s AI stack stable enough for production? What’s the deprecation policy for this package (still pre-1.0 at v0.14)?
   • Monitor Symfony AI’s roadmap for Laravel compatibility improvements.
6. Alternatives:
   • Should we evaluate direct Ollama API clients (e.g., olliwood/ollama-php) or Laravel-specific AI packages (e.g., beberlei/ai)?
   • Compare maintenance overhead vs. feature parity (e.g., streaming, structured outputs, usage/cost metadata).
7. Security:
   • How will we secure Ollama’s API endpoint (e.g., auth, rate limiting)?
   • Should we use Laravel middleware to validate requests before forwarding to Ollama?
8. Cost vs. Benefit (Now Measurable):
   • With per-model token usage available, we can finally run the ROI comparison of self-hosting vs. cloud APIs (e.g., OpenAI) with real numbers rather than estimates.
   • Measure latency, cost per request, and model performance in A/B tests; feed results into a quarterly build-vs-buy review.

Integration Approach

Stack Fit

• Best Fit: Laravel applications requiring Ollama integration for:
  • Real-time chat/assistants (streaming NDJSON responses).
  • Embeddings/vector search (local LLMs for hybrid search).
  • Structured outputs (JSON/arrays for programmatic use).
  • Multimodal AI (audio capabilities with gemma:2b).
  • Offline/edge AI (low-latency, air-gapped systems).
  • Usage-metered / billable AI features (new: v0.14.0 TokenUsage.model enables per-model cost attribution and quota enforcement).
• Workarounds for Laravel:
  • Option 1: Minimalist HTTP Wrapper (Recommended for MVP)
    • Use the package only for HTTP calls, ignoring Symfony abstractions.
    • Replace HttpClient with Laravel’s Http facade or Guzzle.
    • Example:

      use Symfony\Component\AI\Ollama\OllamaClient;
      use Illuminate\Support\Facades\Http;
      
      class OllamaService {
          public function __construct() {
              $this->client = new OllamaClient(
                  Http::macro('createClient', fn() => Http::client())
              );
          }
      
          public function chat(string $model, string $prompt) {
              return $this->client->chat($model, $prompt);
          }
      }
      

  • Option 2: Provider Abstraction (For Advanced Routing)
    • Implement a Laravel-compatible ProviderInterface to route models dynamically.
    • Example:

      interface OllamaModelProvider {
          public function getModel(string $name): string;
      }
      
      class LaravelOllamaProvider implements OllamaModelProvider {
          public function getModel(string $name) {
              return config("ollama.models.{$name}");
          }
      }
      

  • Option 3: Full Symfony Abstraction (High Effort)
    • Abstract Symfony’s AI component in Laravel (e.g., custom AIServiceProvider, interfaces).
    • Useful for long-term Symfony alignment but overkill for simple use cases.
  • Option 4: Usage Ledger Wrapper (New, incremental on Option 1)
    • Wrap calls in a decorator that extracts TokenUsage, reads model, computes cost from a config price table, and records to a Laravel model/job.
    • Example:

      class MeteredOllamaService {
          public function chat(string $model, string $prompt): mixed {
              $response = $this->inner->chat($model, $prompt);
              $usage = $response->getTokenUsage(); // v0.14.0: includes ->model
      
              AiUsage::create([
                  'tenant_id' => tenant()->id,
                  'model'     => $usage->model,
                  'prompt'    => $usage->promptTokens,
                  'completion'=> $usage->completionTokens,
                  'cost'      => config("ai.prices.{$usage->model}.per_1k", 0)
                                 * ($usage->promptTokens + $usage->completionTokens) / 1000,
              ]);
      
              return $response;
          }
      }
      

Migration Path

1. Phase 1: HTTP-Only Integration (1–2 weeks)
   • Replace HttpClient with Laravel’s Http facade.
   • Test basic endpoints (chat, embeddings).
   • Implement error handling (retries, fallbacks).
2. Phase 2: Provider & Catalog Abstraction (1–2 weeks)
   • Introduce OllamaModelProvider / interfaces; move model names into config/ollama.php.
   • Add a scheduled command to sync the model catalog and warm caches.
3. Phase 3: Streaming & Async (2–3 weeks)
   • Move long-running/streamed calls onto Laravel Queues + Horizon; wire broadcasting for UI.
   • Define a post-stream usage capture hook so v0.14.0 cost data is not lost on streamed responses.
4. Phase 4: Cost Telemetry & Metering (1–2 weeks, new)
   • Add ai_usage migration + Eloquent model, price table in config, aggregation command, and a dashboard view.
   • Backfill price config for all catalog models; validate model-name normalization.
5. Phase 5: Guardrails & Rollout (1 week)
   • Budget alerts/throttles, per-tenant quotas, and A/B test harness for cloud-vs-local comparison.

Compatibility

• Package version: Upgrading 0.13 → 0.14 is expected to be non-breaking. Verify no reliance on TokenUsage array shape or json_encode output in custom code; the new model property must be handled if we destructure or persist DTOs.
• Laravel/PHP: PHP 8.1+ / Laravel 9+ baseline unchanged. No new transitive dependencies introduced by the release notes.
• Data contracts: Any existing usage-logging (if already prototyped) needs a nullable model column or backfill default; plan for rows predating the upgrade.
• Deprecations: None announced in v0.14.0. Still pre-1.0, so treat minor bumps as potentially breaking at the margins and pin versions.

Sequencing

• Do not block Phase 1 on cost work; ship the HTTP wrapper first, then layer metering.
• Land the price table config and model-name normalization before enabling billing-facing features, since incorrect model keys silently zero out cost.
• Sequence cost telemetry ahead of any cloud-vs-local ROI decision — it is the evidence base for that call.
• Keep the 0.14 upgrade as its own small PR with regression tests on streaming + usage parsing.

Operational Impact

Maintenance

• Upstream cadence: The package is moving fast (0.7 → 0.8 → 0.13 → 0.14 in a short window). Budget ~1–2 engineer-days per minor release for changelog review, adapter regression tests, and catalog sync.
• Adapter surface: Keeping Symfony touch-points behind our own interfaces confines upgrade work to one or two classes rather than app-wide.
• Price table upkeep: v0.14.0 makes cost a maintained artifact — assign clear ownership for updating per-model rates (and notional rates for self-hosted models) as part of the release checklist.
• No new maintenance burden from v0.14.0 itself, but it introduces a new data asset (usage ledger) that needs retention policy, schema evolution, and GDPR/tenant-deletion handling.

Support

• Runbooks: Add procedures for "Ollama unreachable", "model missing from catalog", "usage rows not appearing", and "cost spike anomaly".
• Observability: Emit metrics for request latency, token counts by model, error rate, and computed cost per tenant; alert on cost-per-request deviations (a strong early signal of model routing mistakes or prompt regressions).
• Team ramp-up: Streaming and Symfony-interop remain the two knowledge cliffs; the cost ledger is comparatively straightforward Laravel work and is a good onboarding task.
• Vendor risk: Pre-1.0 dependency with a fast-moving upstream — maintain a documented fallback (direct Ollama HTTP client) that can be swapped in behind the same interface.

Scaling

• Ollama tier: GPU-bound; scale horizontally with Kubernetes and route by model affinity. Model load time dominates cold-start, so keep hot models resident and pre-pulled.
• Laravel tier: Streaming is the pressure point — use Horizon workers, cap concurrent streams per node, and avoid holding PHP-FPM workers open on long generations.
• Usage ledger volume: At high request rates, per-request inserts become a write hotspot. Batch via queued aggregation (per-minute rollups) or write to a time-series/analytics store; keep raw rows for audit with a TTL.
• Cost computation is cheap (arithmetic on already-parsed usage) — do it at ingest, not at query time.

Failure Modes

Ramp-Up

• Week 1–2: HTTP-only integration behind an internal interface; error handling and health checks; pin 0.14.*.
• Week 3–4: Model provider/catalog config + scheduled sync; basic observability.
• Week 5–6: Streaming via queues/broadcasting; validate post-stream usage capture.
• Week 7–8: Usage ledger, price table, dashboard, budget alerts — first real cost-per-request numbers.
• Week 9+: A/B test local vs. cloud on latency/cost/quality; decide on Option 2 vs. Option 3 abstraction depth for the long term.
• Skill requirements: 1 backend engineer comfortable with Laravel queues/streaming, 1 platform engineer for Ollama/GPU infrastructure, and a part-time owner for cost/pricing governance.