Langconv Laravel Package

Getting Started

LangConv is not a Laravel package — it is a plain Composer library (wikimedia/langconv, GPL-2.0) that implements MediaWiki's Chinese script-conversion engine. There is no service provider, config file, facade, or Blade directive. Assume you are writing the integration layer yourself.

1. Install and confirm the surface area:

composer require wikimedia/langconv
php -r "require 'vendor/autoload.php'; var_dump(class_exists('Wikimedia\\LangConv\\ReplacementMachine'));"

Expect the namespace Wikimedia\LangConv, with the important classes being ReplacementMachine (the converter you call) and FST / FST\Builder (the finite-state transducer used to compile and execute the rule tables). There is also a JSON rule-table directory in the repo (*-t-*.json, e.g. zh-hans-t-zh-hant — "from variant X to variant Y").

2. Read these three things first, in order:

• src/ReplacementMachine.php — the only class you will call on a request path. Its constructor/loading signature is the thing you must pin down against the exact version you install; the PHP and JS builds of this library are not API-identical, and the repo is a Gerrit mirror with a small audience.
• src/FST.php — explains why loading is expensive (the tables are compiled into a transducer at load time).
• The rule-table directory — this tells you which variant pairs actually ship. Do this before you design a UI variant switcher; you may find not every pair you want exists.

3. First useful use case — a one-off in tinker, proving conversion works on real copy pulled from your database:

$machine = app(\App\Services\LangConv::class)->machine('zh-hans', 'zh-hant');
dump($machine->convert('这个软件的内存占用很高'));

Then immediately wrap it: loading/compiling a table per request will be the single biggest performance mistake you can make with this library (see Gotchas).

Implementation Patterns

Bind it as a singleton, keyed by variant pair. The compile cost is real, and Octane/RoadRunner will happily keep it warm for you.

// app/Providers/LangConvServiceProvider.php
public function register(): void
{
    $this->app->singleton(LangConv::class, fn () => new LangConv(
        rulePath: config('langconv.rules_path'),
        defaultVariant: config('langconv.default', 'zh-hans'),
    ));
}

final class LangConv
{
    /** @var array<string, ReplacementMachine> */
    private array $machines = [];

    public function convert(string $text, string $from, string $to): string
    {
        if ($from === $to || $text === '') {
            return $text;
        }
        return $this->machines[$from.'>'.$to] ??= $this->load($from, $to)
            ->convert($text);
    }
}

Resolve the viewer's variant once, in middleware. Priority order that works well in practice: explicit ?variant= → authenticated user preference column → cookie → Accept-Language (zh-TW → zh-tw) → site default. Validate against a hard-coded whitelist; do not pass user input into the machine.

Convert at render, store the source. Keep one canonical string in the database (whatever the author typed or the import produced). Convert in an accessor/presenter or a Blade component. This keeps edits, diffs, and re-conversion sane, and lets you fix rules later without a data migration.

<x-prose :text="$post->body" :variant="session('zh_variant', 'zh-hans')" />

Cache rendered variants for expensive content. Conversion is CPU-bound on long articles; cache the output, not the machine.

$rendered = Cache::remember(
    "post:{$post->id}:{$variant}:v{$post->updated_at->timestamp}",
    now()->addDay(),
    fn () => $converter->convert($post->body, 'zh-hans', $variant),
);

Use it for search recall. This is the highest-value trick for a zh site: normalise both the indexed column and the incoming query to one hub variant (zh-hant or zh-hans) before hitting your full-text index. Users typing simplified to find traditional-titled content is a very common complaint, and this solves it in a few lines.

Pre-compute for large corpora. For a wiki-style import or thousands of posts, put conversion in a queued job that writes to a post_variants table (post_id, variant, body) or a JSON column. Requests then never touch the machine.

Custom glossary layer. Brand names, product terms, and internal jargon will be converted "wrong" by the generic tables. Keep a small, ordered override table of your own and apply it as a pre- or post-pass — this mirrors how MediaWiki wikis maintain their own conversion tables on top of the shipped ones. If your glossary grows large, you can compile it through FST\Builder itself.

Segment before converting. Never run the machine over raw HTML. Split on <code>, <pre>, <script>, URLs, emails, and HTML attributes, convert only the text nodes, and reassemble. If you are rendering Markdown, convert after parsing and before escaping, so you never touch entity syntax.

Gotchas and Tips

• GPL-2.0. This is a copyleft licence. Linking it into a closed-source SaaS is a legal question, not a technical one — get it reviewed before you build a product feature on it. It is also the main reason this library has near-zero adoption outside MediaWiki.
• The GitHub repo is a read-only mirror. Stars (2) and dependents (0) reflect that, not quality. Issues and patches go to Gerrit (per the repo description); do not expect GitHub PRs or issues to be answered. Pin a specific commit or tag in composer.json and read the source of the version you pinned.
• It is the replacement machine, not the markup parser. MediaWiki's familiar -{zh-hans:…; zh-hant:…}- syntax is handled by MediaWiki core's LanguageConverter, not by this library. If you are importing wiki text containing -{}-, -{}- you must implement that layer yourself (strip it, or port the small parser) before feeding text to the machine.
• Load once, per process. Compiling tables per request is the classic mistake; so is instantiating a machine per model in a @foreach. Under Octane, verify memory across workers — the compiled structures are not tiny, and a per-variant map of machines multiplies that.
• Conversion is not round-trip safe. zh-hans → zh-hant → zh-hans will not always return the original bytes, and regional variants (zh-cn, zh-sg, zh-tw, zh-hk, zh-my) deliberately use different vocabulary (软件 / 軟體 / 軟件). Pick a canonical source variant, document it, and never let conversion rewrite stored content.
• Check which pairs ship. If the table you want (zh-hans-t-zh-hk, say) is absent, chain through a hub variant (zh-hant) rather than assuming symmetric support.
• UTF-8 and mbstring are mandatory. Rules match multi-byte sequences; anything already mangled by a Latin-1 import will produce silent no-ops rather than errors. Validate encoding on ingest.
• Longest-match, not word-aware. Rules are greedy over character sequences. That is usually what you want, but it occasionally bites around mixed CJK/ASCII boundaries — test strings like C++軟體v2.
• Do not assume it generalises. MediaWiki also has Serbian, Kurdish, Tajik and Inuktitut converters, but they are much simpler character maps in core and are not what this library ships tables for. LangConv's machinery is generic; the data is Chinese.
• Debugging: when a phrase converts oddly, the fastest path is to compare against a MediaWiki page rendering the same text, then bisect your input line-by-line to find the offending rule. Log the input, output, and variant pair around any user-visible conversion — "it looked fine in Chinese" is not a bug report.
• Extension point: FST\Builder lets you compile your own rule tables into the same JSON format the library consumes. That is the supported way to add a domain glossary without forking, and it keeps your overrides in version control alongside the code that applies them.