Xunsearch Laravel Package

Getting Started

Xunsearch (迅搜) is a standalone Chinese search daemon written in C/C++ (Xapian + SCWS), plus a pure-PHP SDK (hightman/xunsearch) that talks to it over TCP. Nothing in the SDK runs search locally — you must run the server.

1. Install and start the server (once per environment; not via Composer):

wget http://www.xunsearch.com/download/xunsearch-full-latest.tar.bz2
tar -xjf xunsearch-full-latest.tar.bz2
cd xunsearch-full-* && ./setup.sh
/usr/local/xunsearch/bin/xs-ctl.sh -b local start   # indexd :8383, searchd :8384

2. Install the PHP client:

composer require hightman/xunsearch

3. Create the project config — this .ini file defines your "project", fields, and server addresses. This is the single most important file to read first:

; config/xunsearch-blog.ini
project.name = blog
server.index  = 127.0.0.1:8383
server.search = 127.0.0.1:8384

[id]
type = id

[title]
type = title
index = both

[content]
type = body
index = both

[created_at]
type = date
index = self

4. First use case — index and query:

use XS;
use XSDocument;

// In AppServiceProvider::register() — pin the config path explicitly:
XS::loadConfig(base_path('config/xunsearch-blog.ini'));
$this->app->singleton(XS::class, fn () => new XS('blog'));

// Index
$doc = new XSDocument;
$doc->setCharset('UTF-8');
$doc->setFields([
    'id'         => $post->id,
    'title'      => $post->title,
    'content'    => strip_tags($post->body),
    'created_at' => $post->created_at->timestamp,
]);
$xs->index->update($doc);   // add() = new, update() = upsert by id

// Search
$search = $xs->search;
$search->setQuery('搜索引擎')->setLimit(0, 20);
$docs  = $search->search();
$total = $search->getLastCount();

Look at XSDocument, XSIndex, and XSSearch first — they are the whole API surface. The official docs at xunsearch.com (Chinese) are the only reliable reference; the Composer package ships almost no README.

Implementation Patterns

Service provider + typed wrapper. Don't let XS leak into controllers. Bind it and expose an application-level repository:

$this->app->singleton(SearchEngine::class, fn ($app) =>
    new SearchEngine(new XS(config('xunsearch.project')))
);

Model observers for sync. Indexing is a socket write, so keep it in an observer but push the work to a queue:

Post::observe(PostObserver::class);

class PostObserver
{
    public function saved(Post $post)    { SyncPostToIndex::dispatch($post->id); }
    public function deleted(Post $post) { RemovePostFromIndex::dispatch($post->id); }
}

Full reindex command. This is the workflow you'll actually run daily. Batch mode is essential — per-document round-trips are slow:

public function handle(XS $xs)
{
    $index = $xs->index;
    $index->beginBatch();
    Post::chunkById(500, function ($posts) use ($index) {
        foreach ($posts as $post) {
            $doc = new XSDocument;
            $doc->setCharset('UTF-8');
            $doc->setFields($this->map($post));
            $index->update($doc);
        }
    });
    $index->endBatch();
    $index->flushIndex();   // force the daemon to commit
}

Upserts vs inserts. update() replaces the document with the same id; add() inserts and lets the engine assign an ID if id is empty. Use update() for model sync, add() only for append-only logs.

Pagination. setLimit($offset, $limit) — offset first. Wrap it in your own API so it matches Laravel's perPage semantics:

$search->setLimit(($page - 1) * $perPage, $perPage);
$items = collect($search->search())->map(fn ($d) => $d->getFields());
$total = $search->getLastCount();

Highlighting / snippet rendering — do it server-side, not with str_replace:

foreach ($docs as $doc) {
    $doc->setCharset('UTF-8');
    $snippet = $doc->highlight('content', '<mark>', '</mark>');
}

Sorting and facets. Sorting requires the field to be indexed as numeric, date, or string with index = self. Facets (e.g. tags, categories) use a string field plus setFacets():

$search->setSort('created_at', XSSearch::SORT_DESC);
$search->setFacets(['category']);
$search->search();
$facets = $search->getFacets('category');

Query building. setQuery() parses Xunsearch syntax (title:foo AND bar). Add setFuzzy() for typo/partial tolerance and setCollapse()? no — use setFuzzy(false) when exactness matters. Always escape user input that should be literal.

Where to test. Run the daemon in a container/VM in CI and hit it with integration tests — the SDK is thin enough that mocking XS gives you almost no confidence.

Gotchas and Tips

The daemon is not optional. Every new XS() opens a TCP socket to indexd/searchd. Forgetting to start it (or a fresh dev machine) produces XSException: Connection refused deep in a request. Add a health check and fail fast in your provider.

Config discovery is fragile. new XS('blog') searches XS_APP_ROOT/etc/blog.ini by default. Define define('XS_APP_ROOT', base_path()) early, or better, pass the absolute ini path: new XS(base_path('config/xunsearch-blog.ini')). Never rely on the working directory.

Index writes are buffered — search lags. add/update/del queue changes; the daemon commits on an interval. In tests and after bulk loads call $index->flushIndex(), otherwise your "created then searched" assertions fail intermittently.

type = id field is mandatory and must be unique. It's the upsert/delete key. If you leave it out, update() silently behaves like add() and you get duplicates. Keep IDs stable (DB primary keys are fine; ULIDs work but sort oddly as strings).

index = both vs self vs none. A field with index = none is stored but not searchable — a common source of "why does setSort() ignore my field?" errors. Numeric/date sorting requires index = self (or both).

Fuzzy matching defaults trip people up. Recent versions enable fuzzy matching; if a search for "C++" returns unrelated Chinese docs, explicitly call setFuzzy(false). Verify against your installed version — the default has changed across releases.

ID and numeric fields sort as strings unless typed. Declare type = numeric or type = date; a type = string field holding "10" will sort before "9".

Charset. The engine speaks UTF-8. Set $doc->setCharset('UTF-8') and ensure your DB connection is UTF-8 — otherwise Chinese tokens get mangled into garbage without an error.

Chinese segmentation is the point, but it's opinionated. SCWS handles Chinese well; other languages and mixed tokens behave differently (English is lowercased/whitespace-tokenized). If you need custom tokenization, setTokenizer() and SCWS dictionaries are your extension points.

Custom XSDocument subclasses let you centralize field mapping (getFields() override) — much cleaner than repeating setFields() arrays across observer, command, and job.

Don't wrap XS in a long-lived singleton across queue workers unless you handle reconnects; socket state can go stale after the daemon restarts. Reconnect on XSException in a retry block.

Version pinning. hightman/xunsearch follows the server version loosely; the ini format and default fuzzy behavior differ between 1.3.x and 1.4.x. Pin the Composer version and document the matching server build in your Dockerfile.