Yetisearch Laravel Package

Getting Started

If you are already running YetiSearch, this is a drop-in upgrade from 2.6.0 — no config changes, no new migrations. If you are coming from 2.5.x, read the Gotchas section before your next write; the point release is really "upgrade from 2.5.x safely" tooling.

First steps for a new install:

1. Publish config and run the install so the SQLite database and {index}_meta tables exist.
2. Create an index with explicit settings — do not rely on defaults for anything you care about:

YetiSearch::createIndex('articles', [
    'external_content' => true,   // default schema; content lives in {index}_content
    'enable_spatial'   => false,  // skip if you never filter by location
    'stemming'         => 'english',
    'prefix'           => '2 3',
    'detail'           => 'full',
]);

3. Index one record and search it — that is the whole loop:

YetiSearch::index('articles', $article->id, [
    'title'   => $article->title,
    'content' => $article->body,
    'route'   => route('articles.show', $article),
]);

$results = YetiSearch::search('articles', 'anything');

Where to look first:

• config/yetisearch.php for external_content, enable_spatial, chunking and auto_flush.
• The README's stemming section — YetiSearch::stemmingFor($index) is documented there and is now the cheapest way to confirm what an index actually does.
• Your {index}_meta table. In 2.6.1 it records fts_built_by. If that key is missing on an existing index, your next write will rebuild the FTS table for you.

First genuinely useful use case: a search box over an Eloquent model, wired through a model observer that calls index() on save and delete() on delete, with enable_spatial => false unless you need geo filtering.

Implementation Patterns

Treat writes as the only thing that heals an index. 2.6.1's self-healing path triggers on insert(), insertBatch(), update(), delete() and deleteByIdPrefix() — never on search. A practical upgrade pattern:

// after deploying 2.6.1 over a 2.5.x database
// do NOT trust a search to tell you the index is healthy; do one real write.
YetiSearch::update('articles', $anyExistingId, $fields);
// or, more predictably, rebuild deliberately:
YetiSearch::rebuildFts('articles');

Wrap multi-document work in your own transaction. As of 2.6.1, insert(), insertBatch(), update(), delete() and deleteByIdPrefix() join an open transaction via a savepoint instead of throwing There is already an active transaction. This makes the observer/queue pattern safe:

DB::transaction(function () use ($articles) {
    foreach ($articles as $article) {
        YetiSearch::update('articles', $article->id, $this->toDocument($article));
    }
});

Your transaction decides atomicity; a failing write only undoes its own savepoint. Caveat: if you roll back, the "this index is healed" key was never committed, so the next write heals again — correct, just not free.

Prefer batches for bulk work, and know the duplicate rule. insertBatch() and indexBatch() resolve a repeated id by writing only the last group — the chunks and then the document. So [docA_long, docA_short] in one batch indexes docA_short only. This makes idempotent re-imports safe, but it means batch order within a batch matters.

Let the indexer chunk, and re-index whole documents. Chunk deletion of stale chunks is keyed on {id}#chunk{N} and handled transactionally by insert()/insertBatch()/update()/index()/indexBatch()/update(). So the safe pattern is always "write the whole document again", never "append to a document". A #chunk-bearing id of your own is safe — the range query is bounded by id >= '{id}#chunk' AND id < '{id}#chunl' and does not use LIKE, so _ and % in ids are literal.

Check stemming before you debug relevance.

YetiSearch::stemmingFor('articles');  // 'english' | 'french' | 'german' | 'spanish' | null

Returns null when the index does not stem or does not exist — those are different problems and this API will not distinguish them for you. Note it delegates to storage only if the storage implements ProvidesStemming; a custom storage that does not will always report null.

Queue/flush pattern. With auto_flush off, batches are written by flush(); those writes go through insertBatch(), so they inherit both the duplicate-id resolution and the savepoint behaviour.

Deletion. Use delete() for documents and deleteByIdPrefix() for bulk. Since 2.6.1 both share one routine that skips tables that are not there, so indexes built with enable_spatial => false (or with spatial tables dropped by hand) delete cleanly. A location-filtered search on such an index still fails — that is intended.

Gotchas and Tips

• Searches never heal. If an upgrade left you on a 2.5.x-corrupted index, the first search for a common word like title can still throw SQLSTATE[HY000]: General error: 11 database disk image is malformed. It is the next write that fixes it. In a read-only deploy, run rebuildFts() explicitly during release.
• The first write after upgrade can be slow and will take a write lock. The heal rebuilds the FTS table from stored documents with the index's own prefix, detail and stems, inside a transaction that takes the write lock before re-checking fts_built_by. Two connections racing: the second waits, then finds the key set. A connection that cannot get the lock fails the write rather than proceeding — so do not run the heal inside a tight lock-contention window; do it in a maintenance step.
• Do not put the heal inside a long caller transaction. The rebuild runs in your transaction if one is open and remembers nothing until commit. Roll back and the next write heals again. That is safe but repeated.
• You cannot detect the corruption by inspection from the application layer. The fts_built_by key lives in {index}_meta; an index with no {index}_meta table at all does not declare how it is stored and is left to createIndex().
• deleteByIdPrefix() from an old 2.5.x binary re-corrupts the index if run against the same database. The key cannot tell. Ensure every process touching the SQLite file is on 2.6.1.
• external_content => false was never affected — FTS5's rebuild on an own-content table reads its own columns. On such an index the write only sets the key; no rebuild. Do not expect the same latency profile as external content.
• YetiSearch Pro uses a different key (yetisearch_pro_fts_built) and is untouched by all of this. Do not hand-edit or delete either key.
• Failed writes forget what they knew about the key, and every failed write drops the cached key — so a burst of failures can cause repeated heal attempts. Watch for that in logs if writes are erroring.
• Windows/PHP builds without SQLite math functions take a different search path that joins the spatial table. 2.6.1 skips that join when the spatial tables are absent and there is no location filter — but a location search on an index without spatial tables still fails. Guard those code paths.
• stemmingFor() returns the language as given, lower-cased, when there is no stemmer for it. Do not use it as a boolean "is this index stemmed" check unless you control the configured value.
• Extension points: the transactional/savepoint behaviour and the stale-chunk removal are implemented in storage, so a custom storage implementation must reproduce them. If you have a custom storage, verify: (a) it joins an existing transaction via savepoint, (b) it removes {id}#chunk* ranges on rewrite, (c) it resolves duplicate ids in a batch to the last group, and (d) delete() skips absent spatial/id-map tables.
• Debugging checklist for a phantom hit after re-indexing with shorter content: confirm you were on 2.6.1 at write time (stale chunks are only cleaned by a write through insert()/insertBatch()/update()/index()/indexBatch()/update()); confirm the document is not marked is_chunk in its metadata — a document the indexer wrote as a chunk looks for nothing and skips cleanup.
• clear() and rebuildFts() both set the key, so a post-upgrade clear() + re-import is a clean, predictable reset path and often simpler than reasoning about the heal.