Getting Started

Two commands get you running, but treat the second as a deployment step, not a local one:

composer require directorytree/privacy-filter
php artisan privacy-filter:install

privacy-filter:install pulls two artifacts: a compiled privacy-filter.cpp binary matched to the current OS/architecture, and a GGUF model (the default f16 build is multiple gigabytes) from Hugging Face. Both land in storage/app/privacy-filter/ by default. The installer is idempotent — if either file already exists it is silently left alone, so use --force when you actually mean to upgrade.

Publish the config immediately, even if you never edit it:

php artisan vendor:publish --tag=privacy-filter-config

Read these four things, in this order:

config/privacy-filter.php — paths.binary, paths.model, process.timeout, model.url, release.version. Everything else is tuning.
DirectoryTree\PrivacyFilterClassifier\Entity — the entire return contract: type, text, start, end, score, length().
DirectoryTree\PrivacyFilterClassifier\EntityType — the known-vocabulary enum (PrivateEmail, etc.).
PrivacyFilter::fake() — read this before writing your first test, not after.

Verify the install worked before writing any application code:

php artisan tinker
>>> DirectoryTree\PrivacyFilter\Facades\PrivacyFilter::entities('Contact John Doe at [email protected].');

If that returns entities, the binary executes and the model loads. If it hangs, the model is loading (first call is cold). If it throws, check the paths and the executable bit on the binary.

The first use case to build is not user-facing redaction. It is a service that scrubs a log payload or an error-tracker context array before it leaves your app. No latency budget, no UX consequences, no un-redaction requirement, and it removes the highest regulatory exposure per engineering hour of anything on your list. Prove the binary runs, prove the model loads, prove detection quality on your actual log data — then consider putting it on a request path.

use DirectoryTree\PrivacyFilter\Facades\PrivacyFilter;

$entities = PrivacyFilter::entities('Contact John Doe at [email protected].');

foreach ($entities as $entity) {
    $entity->type;  // 'private_email' — raw string from the binary
    $entity->text;  // '[email protected]'
    $entity->start; // 20 (byte offset)
    $entity->end;   // 36 (byte offset, exclusive)
    $entity->score; // 0.98
}

Implementation Patterns

Wrap it behind your own interface on day one

The package has zero dependents and a two-release history (v1.0.0 → v1.1.0, which only added Laravel 13 support). That's fine for adoption, but it means the facade should not appear in fifty call sites. Define a contract and bind the package to it:

namespace App\Support\Redaction;

interface Redactor
{
    /** @return \DirectoryTree\PrivacyFilterClassifier\Entity[] */
    public function entities(string $text, ?float $threshold = null): array;

    public function redact(string $text, string $replacement = '[redacted]', ?float $threshold = null): string;
}

Bind it in a service provider. Swapping to Presidio, AWS Comprehend, or a hosted DLP later becomes one class rewrite instead of a codebase-wide search. This costs fifteen minutes now.

Redact by byte offset, never by `str_replace`

The README's redaction example uses str_replace($entity->text, '[redacted]', $text). It is a readable demo and a bug in production: it replaces every occurrence of the matched substring — including the ones the classifier explicitly did not flag — and it throws away the offsets the binary went to the trouble of producing.

Redact by offset, applied in reverse so earlier offsets stay valid:

public function redact(string $text, string $replacement = '[redacted]', ?float $threshold = null): string
{
    $entities = $this->entities($text, $threshold);

    // Sort by start, then drop anything overlapping a range we already accepted.
    usort($entities, fn ($a, $b) => $a->start <=> $b->start);

    $kept = [];
    $cursor = 0;

    foreach ($entities as $entity) {
        if ($entity->start < $cursor) {
            continue;
        }
        $kept[] = $entity;
        $cursor = $entity->end;
    }

    // Apply from the end backwards so byte offsets remain valid.
    foreach (array_reverse($kept) as $entity) {
        $text = substr_replace($text, $replacement, $entity->start, $entity->length());
    }

    return $text;
}

substr_replace operates on bytes, which is exactly what start / end / length() are. This is the one place where a byte-oriented API works in your favour.

Build a segment map when redaction must be reversible

If the product shows [redacted] to most viewers but must restore the original for an authorised one, do not destroy the text. Store segments:

$parts = [];
$offset = 0;

foreach ($kept as $entity) {
    $parts[] = ['text' => substr($text, $offset, $entity->start - $offset)];
    $parts[] = ['entity' => $entity->type, 'placeholder' => '[redacted:'.$vaultId.']'];
    $offset = $entity->end;
}

$parts[] = ['text' => substr($text, $offset)];

Persist the segments as JSON alongside a reference to the raw text in a restricted store. Redaction becomes reversible and auditable — a common compliance ask — and "which PII did we catch" becomes a query rather than a regex.

One process per call means batching is your biggest lever

Every entities() call spawns the native binary and loads the GGUF model into that process. For a job that classifies ten fields on one record, ten calls means ten model loads. Where the fields are independent, concatenate with a delimiter that cannot appear in the content, classify once, then map offsets back:

$fields = ['subject' => $ticket->subject, 'body' => $ticket->body, 'notes' => $ticket->notes];

$sep = "\n\x1E\n"; // unlikely in free text
$joined = implode($sep, $fields);

$entities = PrivacyFilter::entities($joined, threshold: 0.6);

// Shift each entity's offsets back into its source field.

Verify on real data that the delimiter does not change detection across the boundary — a name split across two fields can be detected as one entity spanning the separator. Validate before trusting it; the win is large enough to be worth the check.

Put classification on a dedicated queue with a memory budget

The README is explicit about this and it is the single most important operational decision. Each concurrent classification loads its own copy of the model. A 3 GB model with four workers is 12 GB before PHP, Redis, and your database.

// config/horizon.php
'privacy-filter' => [
    'connection' => 'redis',
    'queue' => ['privacy-filter'],
    'balance' => 'simple',
    'processes' => 2,
    'tries' => 1,
    'timeout' => 300,
],

RedactTranscript::dispatch($transcript)->onQueue('privacy-filter');

Set tries to 1. A job that times out because the machine is out of memory should not be retried into the same wall. Use a separate queue rather than the default pool so an import job cannot starve privacy filtering, and vice versa.

Serialize entities to plain arrays before they enter a job payload — do not pass Entity objects through the queue.

Centralise thresholds, because there is no global setting

The default threshold of 0.5 is a parameter default on entities(), not a config key. Per the published config, you can tune paths, timeout, model URL, and release — but not the threshold. That belongs in your adapter:

final class PrivacyFilterRedactor implements Redactor
{
    public function __construct(private readonly float $defaultThreshold = 0.65) {}

    public function entities(string $text, ?float $threshold = null): array
    {
        return PrivacyFilter::entities($text, $threshold ?? $this->defaultThreshold);
    }
}

Then expose it as an env var (PRIVACY_FILTER_THRESHOLD) so you can raise it after an incident without a deploy of application logic. Use a lower threshold for audit/scanning workloads and a higher one for anything that writes to user-visible content.

Test with `fake()` — and separately test the real binary

fake() prevents the binary from being invoked, which is what you want in CI (no multi-gigabyte artifacts, no exec permissions):

use DirectoryTree\PrivacyFilterClassifier\Entity;
use DirectoryTree\PrivacyFilter\Facades\PrivacyFilter;

PrivacyFilter::fake([
    '*[email protected]*' => [
        new Entity(
            type: 'private_email',
            start: 20,
            end: 36,
            score: 0.98,
            text: '[email protected]',
        ),
    ],
]);

Two things to be careful about. First, the global (unkeyed) form returns the same entities for every classification, so tests only pass if your redaction logic actually respects offsets — a good forcing function. Second, fake() never exercises the binary, so it will not catch a broken install, a missing model, or drift in the entity vocabulary. Keep a small opt-in integration suite gated behind an env flag:

// tests/Feature/PrivacyFilterIntegrationTest.php
protected function setUp(): void
{
    parent::setUp();

    if (! env('PRIVACY_FILTER_INTEGRATION')) {
        $this->markTestSkipped('Requires installed binary and model.');
    }
}

Run it once per deploy target, not on every commit.

Bake the artifacts into your image, don't download at boot

storage/app/privacy-filter/ is the wrong home for a multi-gigabyte model in most deployments. On ephemeral containers or per-release directory layouts you will re-download it constantly, and the installer's "skip if exists" behaviour means a stale binary can persist across upgrades unnoticed.

Prefer: run privacy-filter:install (or fetch the archive directly) in the Docker build, and point the config at the baked paths:

PRIVACY_FILTER_BINARY_PATH=/opt/privacy-filter/bin/privacy-filter
PRIVACY_FILTER_MODEL_PATH=/opt/privacy-filter/models/privacy-filter-f16.gguf

Pin PRIVACY_FILTER_BINARY_VERSION and, ideally, self-host the GGUF behind your own URL rather than depending on Hugging Face availability at deploy time.

Gotchas and Tips

Offsets are bytes, not characters. start, end, and length() are byte offsets. Any text containing UTF-8 (é, —, emoji, non-Latin names) will have byte offsets that differ from mb_strlen positions. Never mix mb_substr with these values. substr_replace on the raw byte string is correct; anything character-aware is not. Test with a non-ASCII fixture — this is the bug that ships.

Entities can overlap and are not guaranteed to be sorted. A name nested inside a longer address match is plausible. Always sort by start and resolve overlaps before applying replacements, as in the adapter above. Applying unsorted offsets will corrupt the string.

$entity->type and $entity->type() are different things. The property is the raw string from the binary; the method returns the EntityType enum — or null when the binary emits a type the package doesn't know. Code against the enum for branching, but handle null defensively, since that path exists precisely because the model's vocabulary can outrun the package's.

The installer has no upgrade semantics. privacy-filter:install skips existing files silently. A v1.0.0 binary will survive a composer update forever. Either always pass --force in deploy scripts or delete the artifacts first. Also watch for silent arch mismatches — install-binary --url= exists so you can supply a known-good archive when the auto-detection picks wrong.

storage/ is sometimes mounted noexec. The binary will exist, be executable on paper, and still fail to spawn. If you get process spawn errors with correct paths, check the mount options before debugging the package. This is why baking into /opt during the image build is the more reliable route.

Raise process.timeout. The default is 60 seconds. The first invocation on a cold machine has to page in a multi-gigabyte model; a large document adds inference time on top. On constrained hosts, 60s is tight. Set PRIVACY_FILTER_TIMEOUT to something like 120–300 for queued work.

Handle binary failure explicitly. The wrapper shells out; a missing binary, a corrupt model, or an OOM kill surfaces as a process exception, not a domain exception. Catch it in your adapter and decide the policy: for log scrubbing, fail open with a loud alert (losing the log is worse than a leak in the log); for user-facing output, fail closed. Add an artisan health-check command that runs one tiny classification and alerts if it fails, so you find out before an incident does.

Nothing in this package is async. There is no streaming, no batch API, no persistent worker. Every call is a full process spawn plus model load. That is the entire performance model, and it should shape your architecture more than any other detail here.

The model is enormous and its inference is local — that's the point. No text leaves the machine at classification time (only the install reaches the network). That is the strongest reason to choose this over a hosted DLP API, and it is worth stating in your architecture notes. It is also why the memory budget is non-negotiable.

You own the redaction helper. There is no PrivacyFilter::redact() — the README's str_replace snippet is illustrative. Budget for writing and testing your own offset-based redactor, overlap resolution, and reversible segment mapping.

Zero dependents is a signal, not a disqualifier. A single-maintainer wrapper around a binary with one stable release means you should pin the version in composer.json, wrap the facade, and keep the integration test so you notice behavioural drift. Do not let the facade leak into application code.

Privacy Filter Laravel Package