Llphant Laravel Package

Getting Started

LLPhant is a framework-agnostic PHP LLM toolkit (theodo-group/llphant) that works in Laravel without a service provider — you wire it up yourself, which is both the friction and the flexibility.

1. Install and configure keys

   composer require theodo-group/llphant
   

   Add provider keys to .env (OPENAI_API_KEY, ANTHROPIC_API_KEY, MISTRAL_API_KEY, or nothing for Ollama). LLPhant reads these via getenv(), so make sure they're not cached out of reach in a config-only setup (see Gotchas).
2. First call — a chat completion

   use LLPhant\Chat\OpenAIChat;
   use LLPhant\OpenAIConfig;
   
   $config = new OpenAIConfig();
   $config->model = 'gpt-4o-mini';
   
   $chat = new OpenAIChat($config);
   $answer = $chat->generateText('Summarise this support ticket in one line.');
   

3. Where to look first
   • src/Chat/ — one class per provider (OpenAIChat, AnthropicChat, MistralChat, OllamaChat).
   • src/*Config.php — OpenAIConfig, AnthropicConfig, MistralConfig, OllamaConfig, and (as of 1.0.7) HeabsyConfig.
   • src/Embeddings/ and src/VectorStore/ — the RAG building blocks.
   • The examples/ directory in the repo — the fastest way to see a working end-to-end flow.
4. First real use case: a "chat with our docs" endpoint. Load a few markdown files with FileDataLoader, split with TokenTextSplitter, embed into a MemoryVectorStore (or Qdrant/Redis for persistence), then answer with QuestionAnswering. That path touches every core concept in one feature and is usually the reason teams adopt the package.

New in 1.0.7: HeabsyConfig targets the Heabsy API, which is OpenAI-compatible. Use it exactly like OpenAIConfig (same OpenAIChat class, same message/tool APIs) — just set the base URL and key on the Heabsy config. Treat it as "another OpenAI-shaped provider," not a new abstraction.

Implementation Patterns

Bind configs in a service provider, not inline. Constructing new OpenAIConfig() in controllers scatters model names and base URLs across the codebase. Register named bindings:

$this->app->bind('llm.chat.fast', fn () => new OpenAIChat(
    tap(new OpenAIConfig(), function ($c) {
        $c->model = config('ai.fast_model');
        $c->apiKey = config('ai.openai_key');
    })
));

Then inject 'llm.chat.fast' or type-hint a small wrapper of your own.

Wrap LLPhant behind your own interface. Define App\Contracts\LlmClient with generateText() / generateStream() and implement it with LLPhant. This keeps LLPhant types out of your domain, makes swapping to Heabsy/Bedrock/Ollama a config change, and — critically — lets you fake the LLM in tests without hitting the network.

RAG ingestion as a queued job. Document loading, splitting, and embedding is slow and rate-limited. Do it in a job: FileDataLoader → TokenTextSplitter → EmbeddingGenerator → VectorStore::addDocuments(). Key the job by document ID so re-ingestion is idempotent (delete-then-insert vectors, or store a content hash in metadata and skip unchanged chunks).

Persist the vector store for anything real. MemoryVectorStore dies with the request. For production, Qdrant or Redis stores are the usual picks; Doctrine/Postgres works if you're already there. Keep the store binding in the container and pick per-feature if you index multiple corpora.

Conversation memory belongs in your session layer, not the chat object. For multi-turn, pass prior turns via $chat->setSystemMessage() plus a message history you own (session, cache, or DB). LLPhant's chat objects are cheap but not request-scoped state holders — don't serialize them.

Streaming to the browser. Use generateStream() inside response()->stream(), emitting SSE events as tokens arrive, and flush after each chunk. Disable output buffering and set X-Accel-Buffering: no if behind nginx. Always send a terminal event so the frontend knows the stream closed cleanly rather than dying on a timeout.

Structured output for downstream code. Ask for a JSON schema and decode into a DTO rather than regex-ing prose. Validate the decoded array with a Laravel validator or a typed constructor before it touches the DB — models do drift from the schema, and retrying once on validation failure is cheap insurance.

Agents and tools map naturally onto Laravel. Register each tool as a small class with a name, description, and JSON parameter schema; keep tool side effects (DB writes, HTTP calls) idempotent so a retried agent loop doesn't double-write. Log every tool call with the prompt ID — this is your only debugging surface when an agent loops.

Gotchas and Tips

• config:cache and getenv(). LLPhant's config classes commonly fall back to getenv('OPENAI_API_KEY'). Once php artisan config:cache runs, Laravel's .env is no longer loaded, so getenv() returns false. Always set apiKey (and base URL) explicitly from config('services.openai.key') in your bindings — don't rely on the implicit fallback.
• Provider/embedding dimension mismatch. If you swap embedding providers (or models within a provider) you must re-index; the vector store will silently return garbage similarity scores or throw on insert. Store the embedding model name as collection metadata and refuse to query a mismatched collection.
• HeabsyConfig is OpenAI-shaped, but not necessarily OpenAI-identical. Base URL, auth header format, and which endpoints (embeddings, tools, streaming) are supported vary between "OpenAI-compatible" providers. Smoke-test chat, embeddings, and streaming separately before committing a feature to it; a working generateText() says nothing about generateStream().
• Token limits fail late. Long RAG contexts blow the model's window at request time, not at split time. Set a conservative TokenTextSplitter chunk size relative to your smallest model and log the prompt token count on every call so you can see the ceiling approaching.
• No retries or backoff in the library. Rate limits (429) and transient 5xx surface as exceptions. Wrap calls in retry() or a small decorator with exponential backoff; for queued ingestion, $job->release() with a delay beats failing the batch.
• No first-class test doubles. There is no LLPhant::fake(). The interface-wrapper pattern above is the pragmatic answer; alternatively assert on prompts captured by a local stub chat class that implements the same ChatInterface.
• Costs are invisible until the invoice. Log provider, model, input/output token counts, and a request correlation ID per call. A middleware or decorator around your LLM client costs an hour and pays for itself the first time a retry loop runs away.
• Streaming hides errors. Once bytes are flushed you can't change the HTTP status. Emit errors as an SSE event and have the client handle them; catch exceptions inside the stream closure rather than letting them bubble.
• Extension points worth knowing: implement ChatInterface for a bespoke provider, EmbeddingGeneratorInterface for a self-hosted embedding service, and VectorStore for an internal search backend — all are plain interfaces, so Laravel's container resolves them without any package glue.
• Pinning. 1.0.7 is an additive config addition with no breaking changes, but the package moves fast and provider SDKs shift underneath it. Pin a minor version (^1.0) and read the changelog on bumps — config classes are where behaviour quietly changes.