symfony/ai-elasticsearch-store
Elasticsearch vector store integration for Symfony AI Store, adding kNN search and dense_vector support so your Symfony AI apps can store and query embeddings directly in Elasticsearch.
This is a thin bridge package — the README tells you what it does, not how. Expect to spend your first hour in src/.
Prerequisites before touching Composer
dense_vector + kNN search is the whole point, so 7.x works but 8.x is what you want.Install
composer require symfony/ai-elasticsearch-store elasticsearch/elasticsearch
Where to look first
src/ in the package: the Store class (its constructor signature tells you exactly what it needs — client, index name, vector field), plus any StoreFactory / DI extension. This is your real documentation.dense_vector docs. Most "the store is broken" moments are actually mapping mistakes.First use case — a 30-minute smoke test
Spin up Elasticsearch, create one index with an explicit mapping, index three documents, run one similarity query. Do this in a tinker session before writing any application code.
docker run -p 9200:9200 -e "discovery.type=single-node" -e "xpack.security.enabled=false" \
docker.elastic.co/elasticsearch/elasticsearch:8.15.0
Create the index (dimensions must match your embedding model, e.g. 1536 for text-embedding-3-small, 768 for many open models):
$client->indices()->create([
'index' => 'documents',
'body' => [
'mappings' => [
'properties' => [
'embedding' => [
'type' => 'dense_vector',
'dims' => 1536,
'index' => true,
'similarity' => 'cosine',
],
'content' => ['type' => 'text'],
'source_id' => ['type' => 'keyword'],
'tenant_id' => ['type' => 'keyword'],
],
],
],
]);
Then bind the store in a service provider and inject it behind your own interface:
// app/Providers/AiServiceProvider.php
$this->app->singleton(\Symfony\AI\Store\StoreInterface::class, function ($app) {
return new \Symfony\AI\Store\Bridge\Elasticsearch\Store(
$app->make(\Elastic\Elasticsearch\Client::class),
'documents', // index
'embedding', // vector field
// exact constructor args vary by version — verify in src/
);
});
Confirm the class name and argument order against src/ — at 0.x the namespace has moved before. Once a single add() + query() round-trip works, you've de-risked the whole integration.
The single highest-value pattern: put your own interface in front of StoreInterface and never let Symfony\AI\Store\* types leak into controllers or jobs.
interface DocumentIndex
{
public function index(Model $model): void;
public function forget(Model $model): void;
public function search(string $query, array $filters = [], int $limit = 10): Collection;
}
This costs an hour now and saves you when the bridge's 0.x API shifts, or when you want to A/B against pgvector for a subset of queries.
You cannot change dims on an existing dense_vector field. Ever. So:
(embedding model, dimensions) pair: documents_v1_1536.documents_search at it.documents_v2_1536 → backfill → atomically swap the alias → delete the old index.The store writes to a concrete index name, so pass it the write alias and let Elasticsearch handle the cutover.
public function retrieve(string $question, array $filters = []): Collection
{
$vector = $this->embedder->embed($question); // your embedding service
$results = $this->store->query($vector, 10, $filters); // filter support since v0.4
$ids = collect($results)->pluck('source_id'); // store ES doc id => model id
return Document::whereIn('id', $ids)
->get()
->sortBy(fn ($d) => array_search($d->id, $ids->all())); // preserve relevance order
}
Two details that matter: store your Eloquent primary key as a keyword field in the ES document, and re-sort after hydrating — whereIn throws away relevance order and that's the most common silent RAG quality bug.
Don't index inline in the request. Use model observers that dispatch queued jobs:
class DocumentObserver
{
public function saved(Document $doc): void { IndexDocument::dispatch($doc)->onQueue('ai'); }
public function deleted(Document $doc): void { ForgetDocument::dispatch($doc->id)->onQueue('ai'); }
}
Keep a dedicated ai queue with its own worker so a slow Elasticsearch cluster can't starve your mail queue. Set $tries = 3 and backoff() — Elasticsearch transient failures are real, especially on cold clusters.
For backfills, chunk and embed in batches (embedding APIs are the bottleneck, not Elasticsearch). Since v0.14.1, failed operations inside _bulk responses throw rather than being silently tolerated — so wrap the job:
try {
$store->add(...$documents);
} catch (\Throwable $e) {
Log::warning('vector.bulk.failed', ['count' => count($documents), 'error' => $e->getMessage()]);
throw $e; // let the queue retry
}
If you need partial-failure granularity (which specific docs failed), drop to the raw client's bulk() and inspect items[].error yourself.
Build these early — you'll use them constantly:
php artisan ai:index {model} — index one record, useful for debugging.php artisan ai:reindex --fresh — full rebuild, uses clear() (available since v0.11) then re-adds.php artisan ai:stats — _cat/indices doc counts vs. your DB row count, to catch drift.Schedule ai:stats and alert on divergence > 1%. Drift is the #1 operational issue with any vector store.
Unit-test your DocumentIndex implementation against an in-memory/fake store — don't make every test require a cluster. Keep a single integration test tagged @group elasticsearch that runs in CI against a service container, and always index with refresh: true (or call indices()->refresh()) before asserting, otherwise you'll fight flaky empty results.
The documentation is the source code. The README is a stub and every release note since v0.4 says "no significant changes" or a one-line PR title. Budget reading src/ and pinning versions in composer.json with a caret on a specific minor (^0.14), not ^0.4.
Vector dimensions are immutable. Changing your embedding model means a full reindex. Bake the dims into a config value alongside the model name so a mismatch fails loudly at boot rather than producing garbage similarity scores.
Similarity metric must match your vectors. cosine is the safe default for most LLM embeddings. If you use dot_product, Elasticsearch requires unit-normalised vectors — it will accept unnormalised ones and silently rank badly. Normalise explicitly or use cosine.
kNN is approximate. Recall depends on num_candidates in the query. If results "feel wrong" for a known-good query, raise num_candidates and retest before assuming the store is broken. Don't reach for exact search in production unless latency budgets genuinely allow it.
Refresh interval will bite you. Elasticsearch defaults to a 1s refresh. Index-then-immediately-query is the most common false negative in dev and in tests. Pass refresh: 'wait_for' on writes that must be immediately searchable.
Bulk failures now throw (v0.14.1). This is a behavioural change, not a cosmetic one. If you have existing bulk jobs that assumed "it mostly worked", they will now blow up on partial failures — which is what you want, but only if your retry/dead-letter handling exists. Audit bulk paths before upgrading. Also note idempotency: retrying a bulk job must not create duplicate documents — use deterministic _id values derived from your model key.
Base URL handling changed in v0.11. If you front Elasticsearch with a reverse proxy or a managed endpoint that carries a path prefix (e.g. /es), verify the client's configured URL resolves correctly with the index name appended. Symptom: 404s that look like a missing index but are actually a doubled or dropped path segment. Log the full request URI on failure.
clear() is destructive. It's designed for rebuild workflows. Never expose it to a controller, and double-check whether it drops the index (losing your mapping) or deletes documents — behaviour differs across bridges. If it drops the index, make sure your reindex command recreates the mapping before add().
Don't singletons the store if the index varies. Tenant-per-index or environment-per-alias setups need the index name resolved per call. Bind a small factory instead of a singleton on StoreInterface.
Debug with the wire, not the API. When results are wrong, log the actual query body and response. GET /_cat/indices?v, GET /documents/_mapping, and GET /documents/_count answer 90% of "why is this empty" questions in seconds. Compare _count against your DB count first, every time.
Elasticsearch is real infrastructure. If your dataset is under ~100k vectors and you already run PostgreSQL, pgvector is materially less operational surface for the same query semantics. Choose this package when you also want Elasticsearch's text relevance, aggregations, and hybrid scoring — that's where the extra complexity pays for itself.
How can I help you explore Laravel packages today?