Fias Component Laravel Package

Getting Started

composer require liquetsoft/fias-component

This is not a Laravel package — there is no service provider, no config file, and no facade. It is a framework-agnostic toolkit for the three problems FIAS import actually has: fetch the right archive, unpack and stream it, and write the records somewhere. You own the wiring, the storage, and the scheduling.

Read these four namespaces first, in this order:

1. Liquetsoft\Fias\Component\FiasInformer — talks to the FIAS SOAP service. FiasInformer::getLastVersion() returns a version object carrying the version id, publication date, and download URL; getVersionList() gives you the full/delta history.
2. Liquetsoft\Fias\Component\Pipeline — Pipeline (or Pipe), State, and Task. Everything the package does is a Task that reads and mutates a parameter-bag State. This is the mental model for the whole library.
3. Liquetsoft\Fias\Component\Reader — a streaming XML reader. Do not reach for SimpleXML; FIAS files are gigabytes.
4. Liquetsoft\Fias\Component\Storage — the contract you implement to land records in Postgres/MySQL.

First use case — a php artisan fias:check command. Ask the informer what is current, compare it to a fias_versions table you maintain, and exit non-zero if there is an update. Twenty lines, no pipeline yet, and it validates that your SOAP access works before you commit to a 40 GB download:

$informer = new FiasInformerImpl('https://fias.nalog.ru/WebServices/Public/');
$last     = $informer->getLastVersion();   // version id, date, url
$current  = DB::table('fias_versions')->max('version');

if ($last->getVersion() > $current) {
    $this->info("FIAS {$last->getVersion()} available ({$last->getDate()})");
}

Once that works, build a Pipe out of DownloadTask, UnpackTask, ReadTask, and ImportTask and pass it a State populated with version, url, and a temp directory. The pipe runs the tasks in order and hands the same State down the chain — that is the entire execution model.

Implementation Patterns

Run the pipeline as a whole, or break it into resumable stages. For a nightly delta cron the whole pipe is fine. For a full import (which is hours and tens of GB) split it: a DownloadJob that stores the archive, an UnpackJob that explodes it to a temp tree, and one ImportJob per FIAS entity folder (ADDR_OBJ, HOUSES, APARTMENTS, …). Each stage writes a row to fias_imports so a failure at hour four doesn't restart from zero.

Implement Storage against the query builder, not Eloquent. FIAS full imports push tens of millions of rows. Storage::insert() should buffer and call DB::table($t)->upsert($chunk, ['id'], $updateColumns) in batches of 1,000–5,000. Use start()/stop() to open and close a transaction per file, never per record.

Use upsert semantics — FIAS deltas send whole records, not patches. A delta for a record contains every column, so an insert ... on conflict do update is both correct and idempotent. It also means re-running a failed import is safe, which you will need.

Track versions in your own table and apply deltas strictly in order. A fias_versions table (version, date, is_full, applied_at) is not optional. Deltas are only valid applied on top of their immediate predecessor; if you detect a gap, fall back to a full import rather than skipping.

Full import strategy: truncate → drop indexes → COPY/LOAD DATA → rebuild indexes. The package's TruncateTask gets you partway. On Postgres, bypassing the ORM and using COPY for the initial load is roughly an order of magnitude faster than batched inserts.

Register a narrow EntityManager set. ManagerFactory builds entity managers from the FIAS metadata. If you only care about addresses, define managers for three entities and skip the other sixty. The pipeline will simply won't see the folders you don't register.

Decorating is the extension point for observability. The package logs almost nothing. Wrap each Task in your own TimedTask implementation that records duration and row counts into State, then persist the final State as a JSON blob on your fias_imports row. Because State is a plain parameter bag, it serialises cleanly.

Swap in your own Downloader and Reader when the defaults chafe. The bundled downloader uses cURL; a Guzzle wrapper gives you retries, proxies, and S3 multipart. Similarly, a custom Reader can stream from object storage instead of a local temp directory.

Gotchas and Tips

• Never load a FIAS XML into memory. The bundled Reader is XMLReader-based and yields one entity at a time. If you write your own reader, preserve that contract — a single AS_HOUSES file can exceed 2 GB uncompressed.
• The SOAP informer is unreliable and has no SLA. Cache getLastVersion() for a few hours, retry with exponential backoff, and fall back to constructing URLs against the public fias.nalog.ru file endpoints if the service is down during a scheduled run.
• Archive format varies by era. Older FIAS full archives are RAR and the Unpacker shells out to the unrar binary — check it is installed on the host before you schedule the job. Newer GAR releases are ZIP. The failure mode is a silent empty temp directory, so assert the unpacked tree is non-empty before starting the import.
• GUIDs are char(36), not integers. FIAS identifiers are UUIDs. Storing them in varchar(255) bloats the index by an order of magnitude on a 30M-row table.
• Dates arrive as YYYY-MM-DD strings and can be empty. '' in a DATE column will abort the insert on Postgres. Cast to NULL in the Storage layer, not with a database default.
• Sanitise stray NUL bytes. Some historical FIAS dumps contain \x00 in NAME fields, which Postgres rejects in text columns and MySQL truncates at silently. Strip them when reading.
• Single writer, always. Two concurrent importers into the same schema will deadlock on upserts. Take an advisory lock (pg_advisory_lock) or a row in a fias_locks table at the start of Storage::start().
• Disable indexes for full reloads, but not for deltas. Delta imports need indexes to be present, or each upsert does a sequential scan.
• The package throws Liquetsoft\Fias\Component\Exception\*. Catch the base exception at the console-command boundary, log the originating task from State, and mark the import row failed — do not let it bubble into a queue worker's generic retry loop, which will re-run the entire download.
• No queue integration ships with the package. If you want progress reporting, dispatch each entity folder as a separate job and update a percentage column; the State bag is the natural place to hand progress figures back to the job.