Ollama Laravel Laravel Package

Getting Started

• Install: composer require cloudstudio/ollama-laravel — v1.3.0 widens the framework constraint to include Laravel 13, so the same command works on 10.x → 13.x. No migrations, no new config keys, no breaking changes in this release; treat it as a safe minor bump.

• Publish the config: php artisan vendor:publish --tag="ollama-config" → config/ollama.php. This is the first file to read. It holds the Ollama base URL (typically OLLAMA_URL/http://localhost:11434) and the default model name. Every project-specific decision (host, default model, timeouts) belongs here, driven by env().

• Have a model available before you write code. The package is a thin HTTP client, not a model host. ollama serve on the target machine (or a sibling container) and ollama pull llama3.2 — otherwise your first call fails with a model not found error that looks like a package bug but isn't.

• Where to look first, in order: (1) config/ollama.php after publishing, (2) the Ollama facade class and the fluent builder it returns in src/ — that builder is the entire public API surface, (3) the README examples, which show the canonical agent()->prompt()->model()->options()->ask() chain. The README in v1.3.0 also carries a CloudStudio services banner; ignore it, it's marketing, not API.

• First use case — a throwaway artisan command, not a controller:

  use Cloudstudio\Ollama\Facades\Ollama;
  
  $answer = Ollama::agent('You are a terse Laravel expert.')
      ->prompt('Explain route model binding in one sentence.')
      ->model(config('ollama.model'))
      ->options(['temperature' => 0.2])
      ->ask();
  

  Once that returns text, you've validated network, model, and config in one shot. Only then move it behind a service class and into a job.

Implementation Patterns

• Never call the facade from a controller action directly. Wrap it in a single App\Services\OllamaService (or a PromptRunner) that owns the facade, the default model, and the option array. Controllers/jobs depend on your class. This is the difference between a package you can swap out later and one you're married to.
• Keep system prompt and user prompt separate. Use agent() for the persona/instructions (stable, ideally a config string or a prompts/ file) and prompt() for the interpolated user input (variable). Mixing them means every tweak to the persona touches request-handling code.
• Anything over a couple of seconds goes in a job. Generation is slow and non-deterministic; a synchronous call in an HTTP request will time out under load. Dispatch a GenerateSummary job, use ShouldBeUnique keyed on the input hash so double-clicks don't queue the same GPU work twice, and persist the result to a model column the UI polls (or broadcast on completion).
• Streaming for chat-style UX. The builder's stream flag plus a callback is the pattern to reach for when the user is watching the output. Serve it through response()->stream() with X-Accel-Buffering: no and an ob_end_flush()/flush() per chunk. Do not stream inside a queued job — there's no consumer.
• Structured output: ask for JSON, then validate and retry. Instruct the model to emit a single JSON object, json_decode it, run it through a Validator, and on failure re-prompt once with the validation errors appended. Treat the model as an unreliable third-party API — because that's what it is.
• Cache aggressively. Cache::remember('ollama:'.sha1($model.$prompt.$temperature), now()->addDay(), fn () => ...) turns repeated classification/labelling calls into free ones. For prompt sets that are mostly static, this alone can cut GPU load by an order of magnitude.
• Route by task, not by one big model. Keep a config/ai.php map of task → model (a small 3B model for tagging/routing, a larger one for drafting). Cheap tasks on cheap models keeps latency and hardware pressure sane.
• RAG with embeddings. Chunk documents, embed each chunk, store the vectors (pgvector/vector column, or a dedicated index), retrieve top-k by cosine distance, and stuff the results into the prompt. Keep the retrieval layer in your app — the package just gives you the embedding and completion calls.
• Testing: the package rides Laravel's HTTP client, so Http::fake([...]) with a canned Ollama JSON payload gives you fast, deterministic tests of your service, which is where the logic lives. Assert you built the right prompt, not that the model answered well.
• Local/dev parity: a docker-compose service for ollama with the model pulled at boot, OLLAMA_HOST=0.0.0.0 inside the container, and the app's ollama.url pointed at the service name. Warm the model on deploy (curl /api/generate with a tiny prompt) so the first real user doesn't pay the load cost.

Gotchas and Tips

• Timeouts are the #1 support ticket. Long generations routinely exceed the default HTTP timeout. Set an explicit, generous timeout in config/ollama.php (or per-call if the builder exposes it), and cap output length via the model options (num_predict) so a runaway generation can't hang a worker forever.
• Streaming breaks under the usual Laravel middleware stack. Debugbar, response compression (mod_deflate/gzip), Content-Length-setting middleware, and full-page caches will buffer or corrupt the stream. Test streaming with those disabled, and never route streamed responses through a route that's behind cache.response.
• env() only works inside config files. If you're tempted to read env('OLLAMA_URL') in a service class, don't — it returns null the moment someone runs php artisan config:cache in production. Always go through config().
• "Model not found" is an ops problem, not a code problem. The model must be pulled on whichever host ollama.url points at. Add a health/warm-up check to your deploy script rather than debugging it from a stack trace.
• Response shapes differ per endpoint. Generation returns a top-level text field; chat returns the content nested under a message object; embeddings return an array of floats. Don't assume one shape across calls — null-coalesce defensively and log the raw payload in local/testing so mismatches are obvious.
• Single-GPU concurrency queues silently. Ten simultaneous requests don't error, they just get slow — and can look like a timeout bug in your app. Either serialize with a queue and a single worker, or tune OLLAMA_NUM_PARALLEL/OLLAMA_MAX_LOADED_MODELS deliberately.
• Never expose port 11434. Ollama has no auth and no per-model authorization. Bind to 127.0.0.1 or a private network, and if the app must reach it across hosts, put it behind your own authenticating proxy — that proxy URL is then just a config change, which is exactly why you kept the URL in config/ollama.php.
• Extension points worth knowing: the base URL is your swap-out seam (local Ollama → remote Ollama → your own proxy → a different vendor's OpenAI-compatible endpoint); the client/builder class is extendable if you need per-request headers or logging; and you can bind a fake implementation of your own OllamaService interface in the container for tests that shouldn't touch HTTP at all.
• On v1.3.0 specifically: the only functional change is the Laravel 13 constraint in composer.json, so the upgrade risk is essentially a composer update + test-suite run. Pin ^1.3 rather than a wide-open constraint, and if you're on Laravel 12 or below, there's no urgency — but it does unblock a future framework upgrade. The README banner is vendor marketing with no code, config, or telemetry impact; worth a one-line note if your org runs dependency/licence review, but nothing to action.