Privacy Filter Classifier Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require directorytree/privacy-filter-classifier
   

   Ensure your Laravel app has PHP 8.1+ (check php -v in your project).

2. First Use Case: Classify PII (Personally Identifiable Information) in user-generated text (e.g., support tickets, comments, or form submissions). Example:

   use DirectoryTree\PrivacyFilterClassifier\Classifier;
   
   $classifier = new Classifier(
       binaryPath: base_path('vendor/directorytree/privacy-filter/bin/privacy-filter'),
       modelPath: base_path('storage/models/privacy-filter-f16.gguf'),
       timeout: 30
   );
   
   $text = "Email me at [email protected] or call +1-555-123-4567";
   $entities = $classifier->entities($text);
   

3. Where to Look First:

   • Classifier class: Core interface for classification.
   • Entity class: Represents detected PII (e.g., emails, phone numbers).
   • config/privacy-filter.php: If the package adds one (check for future updates).

Implementation Patterns

Workflows

1. Request Validation + Classification:

   use Illuminate\Http\Request;
   use DirectoryTree\PrivacyFilterClassifier\Classifier;
   
   public function store(Request $request, Classifier $classifier) {
       $text = $request->input('comment');
       $entities = $classifier->entities($text, threshold: 0.8);
   
       if ($entities->isNotEmpty()) {
           return back()->withErrors(['pii_detected' => 'Remove sensitive data.']);
       }
       // Proceed with storage...
   }
   

2. Batch Processing: Use Laravel Queues to classify large texts asynchronously:

   $classifier = app(Classifier::class);
   dispatch(new ClassifyTextJob($text, $classifier));
   

3. Model Management:

   • Store models in storage/models/ (e.g., privacy-filter-f16.gguf).
   • Update models via a ModelUpdater service (extend the package if needed).

Integration Tips

• Laravel Service Provider: Bind the classifier to the container for dependency injection:

  $this->app->singleton(Classifier::class, function ($app) {
      return new Classifier(
          binaryPath: $app['path.base'].'/vendor/...',
          modelPath: $app['path.storage'].'/models/privacy-filter-f16.gguf'
      );
  });
  

• Middleware for PII Scanning: Create middleware to auto-scan incoming requests:

  public function handle($request, Closure $next) {
      $textFields = ['comment', 'message', 'description'];
      foreach ($textFields as $field) {
          if ($request->has($field)) {
              $entities = app(Classifier::class)->entities($request->input($field));
              if ($entities->isNotEmpty()) {
                  return response()->json(['error' => 'PII detected'], 400);
              }
          }
      }
      return $next($request);
  }
  

• Entity Formatting: Format Entity objects for APIs or views:

  $entities->map(fn ($entity) => [
      'type' => $entity->type(),
      'value' => $entity->value(),
      'confidence' => $entity->confidence(),
      'start' => $entity->start(),
      'end' => $entity->end(),
  ]);
  

Gotchas and Tips

Pitfalls

1. Binary Path Issues:

   • Ensure privacy-filter binary is in vendor/directorytree/privacy-filter/bin/ or specify a custom path.
   • Debug: Check file_exists($binaryPath) before instantiating Classifier.

2. Model File Permissions:

   • Models in storage/ must be writable by the PHP process:

     chmod -R 755 storage/models/
     

3. Timeout Handling:

   • Default timeout: 60 may be too long for APIs. Reduce to 10 for faster responses (trade-off: fewer detections).
   • Debug: Wrap calls in try-catch to handle timeouts gracefully:

     try {
         $entities = $classifier->entities($text, timeout: 10);
     } catch (RuntimeException $e) {
         Log::error("PII classification timed out: {$e->getMessage()}");
     }
     

4. Threshold Sensitivity:

   • Default threshold (0.5) may return false positives. Test with:

     $entities = $classifier->entities($text, threshold: 0.9); // Stricter
     

5. Memory Usage:

   • Large texts (>10KB) may cause high memory usage. Split text into chunks:

     $chunkSize = 5000;
     $chunks = array_chunk(str_split($text), $chunkSize);
     $entities = collect([]);
     foreach ($chunks as $chunk) {
         $entities->push(...$classifier->entities($chunk));
     }
     

Debugging

• Log Entities:

  $entities = $classifier->entities($text);
  Log::debug('Detected PII:', [
      'entities' => $entities->map(fn ($e) => $e->toArray()),
      'text' => $text,
  ]);
  

• Check Binary Output: Temporarily log raw binary output for debugging:

  $classifier->setDebugMode(true); // If supported (check package docs)
  

Extension Points

1. Custom Entity Types: Extend the Entity class to add metadata (e.g., isSensitive()):

   class ExtendedEntity extends \DirectoryTree\PrivacyFilterClassifier\Entity {
       public function isSensitive(): bool {
           return in_array($this->type(), ['email', 'phone', 'credit_card']);
       }
   }
   

2. Pre/Post-Processing: Wrap the classifier to add logic:

   class PIIClassifier {
       public function __construct(private Classifier $classifier) {}
   
       public function classify(string $text): array {
           $entities = $this->classifier->entities($text);
           // Add custom logic (e.g., redact entities)
           return $entities->filter(fn ($e) => $e->confidence() > 0.8)->values();
       }
   }
   

3. Model Versioning: Store multiple model versions in storage/models/v1/, storage/models/v2/, and switch dynamically:

   $modelPath = match ($request->input('model_version')) {
       'v2' => base_path('storage/models/v2/privacy-filter.gguf'),
       default => base_path('storage/models/v1/privacy-filter.gguf'),
   };
   

4. Caching Results: Cache classification results for identical texts (e.g., using Illuminate\Support\Facades\Cache):

   $cacheKey = "pii:{$text}";
   return Cache::remember($cacheKey, now()->addHours(1), function () use ($classifier, $text) {
       return $classifier->entities($text);
   });