Ollama Php Laravel Package

Getting Started

Prerequisites (do these before touching the package)

1. An Ollama server you can reach: ollama serve locally, or a named host on the internal network. Verify with curl http://localhost:11434/api/tags — if that fails, no amount of PHP will help.
2. At least one generation model pulled: ollama pull llama3.2 (or whatever the model evaluation picks).
3. A separate embedding model pulled if you plan on semantic search: ollama pull nomic-embed-text. Generation models do not double as embedding models.

Install and wire it up

composer require juststeveking/ollama-php

Put the endpoint in .env rather than hardcoding it — you will have a dev host and a prod host before the month is out:

OLLAMA_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2

If the package ships a Laravel service provider, publish its config (php artisan vendor:publish) and drive the URL from there. If it doesn't, bind the client yourself in a ServiceProvider — two lines, and it keeps the endpoint configurable in one place.

First use case: prove the loop, in Tinker

use JustSteveKing\Ollama\Ollama;

$ollama = Ollama::client(url: config('services.ollama.url'));

$response = $ollama->generate(
    model: 'llama3.2',
    prompt: 'List three use cases for local LLM inference in one line each.',
);

dump($response);

That's the whole first milestone: an php artisan tinker session that returns text. Confirm you get a response object back and inspect its shape ($response->body, ->json(), or a typed response DTO — check src/) before writing anything else. You will be reading that shape constantly.

Where to look first in the repo

Because the package is thin, reading all of src/ takes twenty minutes and tells you more than any docs will.

Implementation Patterns

1. Never call the SDK from a controller — put it behind one adapter

This is the single decision that determines whether this ages well. Define the contract before you touch the SDK:

namespace App\Contracts;

interface LlmClient
{
    public function complete(string $prompt, array $options = []): string;
    public function chat(array $messages, array $options = []): string;
    /** @return float[] */
    public function embed(string $text): array;
}

Then one OllamaLlmClient implementing it, with the package imported only in that file. Swapping to a hosted provider later becomes a container binding, not a refactor. It also means the "the maintainer abandoned it" conversation is a 200-line rewrite scoped to one class.

2. Run generation in queued jobs, not in the request cycle

A 300-token generation on a laptop-class GPU is 5–30 seconds. Anything user-facing that waits on it will feel broken and will blow through PHP-FPM timeouts.

class SummariseThread implements ShouldQueue
{
    public int $timeout = 300;
    public int $tries = 3;
    public array $backoff = [10, 60, 180];

    public function __construct(public int $threadId) {}

    public function handle(LlmClient $llm): void
    {
        $thread = Thread::findOrFail($this->threadId);

        $summary = $llm->complete(
            view('prompts.summarise-thread', ['thread' => $thread])->render()
        );

        $thread->update(['summary' => $summary, 'summarised_at' => now()]);
    }
}

Dispatch it, return 202 Accepted or a "generating…" state to the UI. Note the timeout is set well above Laravel's defaults, and the job is idempotent because retries will happen.

3. Chat = persisted history, replayed each turn

The SDK is stateless; Ollama is stateless. Conversation memory is your database table, not the client.

$messages = $conversation->messages()
    ->latest()
    ->limit(20)                     // context window is finite; trim deliberately
    ->get()
    ->reverse()
    ->map(fn ($m) => ['role' => $m->role, 'content' => $m->content])
    ->push(['role' => 'user', 'content' => $input])
    ->all();

$reply = $llm->chat($messages);

Store the rendered prompt and the raw response for every call. When output quality changes after a model upgrade, that log is the only way to diff behaviour.

4. Streaming for anything with a visible UI

Ollama streams newline-delimited JSON. Stream it through to the browser with response()->stream() or an SSE endpoint, and consume the chunked body incrementally rather than collecting it first:

return response()->stream(function () use ($prompt) {
    foreach ($this->ollama->stream($prompt) as $chunk) {
        echo "data: " . json_encode(['delta' => $chunk]) . "\n\n";
        ob_flush(); flush();
    }
    echo "data: [DONE]\n\n";
}, 200, ['Content-Type' => 'text/event-stream', 'X-Accel-Buffering' => 'no']);

The X-Accel-Buffering: no header and the ob_flush() matter more than they look — nginx and PHP's output buffer will happily batch your "real-time" stream into one chunk otherwise. If you're on Octane, test streaming early; it's the most common place this class of feature falls over.

5. Embeddings + retrieval (the pattern most people actually want)

Chunk → embed → store vector → embed query → nearest-neighbour → stuff into the prompt.

foreach ($chunks as $chunk) {
    DocumentChunk::create([
        'content'     => $chunk,
        'embedding'   => $llm->embed($chunk),   // float[]
        'model'       => 'nomic-embed-text',
    ]);
}

Pin the embedding model per row (as above). Vectors from different models are not comparable, and you will change models. Postgres + pgvector is the path of least resistance if you're already on Postgres; sqlite-vss works for a spike.

6. Structured extraction: ask for JSON, then validate it

Do not trust the model's formatting. Ask for JSON, then treat the output as hostile input:

$raw = $llm->complete($prompt, ['format' => 'json']);
$data = json_decode($raw, true, flags: JSON_THROW_ON_ERROR);
$dto = ExtractedInvoice::from(validator($data, $rules)->validate());

Wrap that in a retry-once-with-the-parse-error loop. It resolves the majority of malformed outputs without you writing a JSON repair routine.

7. Test by faking HTTP, not by mocking the SDK

Fake at the Guzzle/PSR-18 boundary so your adapter is exercised for real:

$mock = new MockHandler([new Response(200, [], json_encode(['response' => 'stub']))]);
$client = new Ollama(new Client(['handler' => HandlerStack::create($mock)]), 'http://test');
$this->app->instance(LlmClient::class, new OllamaLlmClient($client));

This gives you fast, deterministic tests in CI where no Ollama server exists. Keep a separate, opt-in "contract test" suite that runs against a real model before each release — that's the only thing that catches API drift.

8. Log every call as a first-class event

Log::info('llm.call', [
    'model' => $model, 'operation' => 'chat',
    'prompt_tokens' => $usage['prompt_eval_count'] ?? null,
    'duration_ms' => $ms, 'status' => 'ok',
]);

You cannot tune timeouts, batch sizes, or model choices without this. It's the cheapest observability you'll ever add.

Gotchas and Tips

Timeouts will bite you first. HTTP client defaults are tuned for web requests, not inference. Set a generous total timeout (120–300s) and a short connect timeout (2–5s) so a dead host fails fast instead of hanging a worker for three minutes. If the package doesn't let you inject your own PSR-18 client, that's the trigger to fork it — retries, tracing, and mTLS all live at that layer.

Cold starts are real. The first request after a restart loads the model into VRAM and can take tens of seconds. Use the keep_alive option to hold the model in memory between requests, and exclude cold-start measurements from your latency percentiles or your numbers will be nonsense.

Error semantics are coarse. Distinguish "connection refused" (host down — infrastructure problem, don't retry in a tight loop), "404 model not found" (you forgot to ollama pull on the new host — config problem, alert loudly), "500 out of memory" (model too big for the GPU — capacity problem), and "timeout" (probably just slow — retry with backoff). Collapsing these into one "LLM failed" exception means every incident starts with an hour of guesswork.

Model tags are exact. llama3 and llama3:8b and llama3:70b are different things and only some will be pulled. Treat the model identifier as a required config value, validated at boot, not a magic string in a service class.

Streaming and format: json are mutually exclusive in practice. If you want a structured object back, ask for a non-streamed response and parse it. Asking for both gives you partial JSON to reassemble.

Ollama serialises requests to a model by default. Ten queued jobs hitting the same model don't run ten-wide; they queue inside Ollama. Tune OLLAMA_NUM_PARALLEL / OLLAMA_MAX_LOADED_MODELS on the host deliberately, and put a rate limiter or per-model concurrency cap on your queue so you're not stacking work you can't drain. This is a host configuration concern, not an SDK one — but it will look like an SDK performance problem when jobs start timing out.

stream: false if you want a complete response. Streaming returns an iterable of chunks; if you forget to disable it, $response will be a stream you then have to concatenate, and someone will inevitably .body it and get one word.

Context windows are finite and truncation is silent. A long thread will be quietly cut, usually from the wrong end. Trim history deliberately (recent-first, or summarise older turns) and assert a rough token budget before sending.

Embedding dimensions differ by model. nomic-embed-text is 768-dim; others differ. If you change models, existing vectors are dead and the column type may not fit the new ones. Store the model name alongside each vector and plan for a full re-index.

Vendor the source or pin hard. Twenty-eight stars, zero dependents, one maintainer. Pin an exact version in composer.json, and if the feature is anywhere near revenue, copy the adapter into your repo. The package is small enough that this costs an hour and removes an entire class of risk.

Ollama ships an OpenAI-compatible endpoint (/v1/chat/completions). If you later want to try a mainstream PHP client without abandoning local inference, that endpoint is your escape hatch. Design your LlmClient contract around your semantics so the choice of transport stays a detail.

Verify feature coverage against your Ollama version, not the README. Tool calling, structured outputs, and vision support have all moved on Ollama's side. Before you promise a capability, hit the raw API with curl and confirm the server supports it, then confirm the SDK surfaces it. A thin wrapper lagging the upstream API is a blocker the moment product asks for the thing it doesn't expose.

Read the src/ before the README. On packages this small, the source is the documentation, and it is more current than any example in the readme — especially method names and option arrays, which are the most likely thing to have drifted.