Ai Vektor Store Laravel Package

Getting Started

Minimal Setup

1. Install Dependencies:

   composer require symfony/ai centamiv/vektor
   

   Ensure your Laravel app uses PHP 8.2+ and has either Redis or PostgreSQL with pgvector installed.

2. Configure Vektor: Add to config/services.php:

   'vektor' => [
       'dsn' => env('VEKTOR_DSN', 'redis://127.0.0.1:6379'),
       'collection' => 'default',
   ],
   

3. Register the Store: Create a service provider (app/Providers/VektorServiceProvider.php):

   use Symfony\Component\AI\Store\VectorStoreInterface;
   use Symfony\Component\AI\Store\VektorStore;
   
   class VektorServiceProvider extends ServiceProvider {
       public function register() {
           $this->app->singleton(VectorStoreInterface::class, function ($app) {
               return new VektorStore($app['config']['services.vektor']);
           });
       }
   }
   

4. First Use Case: Embed and search a query in a Laravel controller:

   use Symfony\Component\AI\Store\VectorStoreInterface;
   
   class SearchController extends Controller {
       public function __construct(private VectorStoreInterface $store) {}
   
       public function semanticSearch(string $query) {
           $embedding = $this->store->embed($query);
           $results = $this->store->search($embedding, limit: 3);
           return response()->json($results);
       }
   }
   

Where to Look First

• Symfony AI Store Documentation for interface methods (embed, search, upsert).
• Vektor README for backend-specific tuning (Redis/PostgreSQL).
• Laravel’s config/services.php for environment variable overrides (e.g., VEKTOR_DSN).

Implementation Patterns

Core Workflows

1. Embedding and Storage

• Pattern: Use upsert for idempotent writes (avoid duplicates):

  $this->store->upsert(
      id: 'doc_123',
      vector: $embedding,
      payload: ['title' => 'Laravel AI', 'content' => '...']
  );
  

• Laravel Integration: Attach to model observers or events:

  // app/Observers/DocumentObserver.php
  class DocumentObserver {
      public function saved(Document $document) {
          $embedding = $this->generateEmbedding($document->content);
          app(VectorStoreInterface::class)->upsert(
              id: 'doc_'.$document->id,
              vector: $embedding,
              payload: $document->toArray()
          );
      }
  }
  

2. Hybrid Search with Scout

• Pattern: Extend Laravel Scout’s Engine to delegate vector queries:

  // app/Scout/Engines/VektorEngine.php
  use Symfony\Component\AI\Store\VectorStoreInterface;
  
  class VektorEngine extends Engine {
      public function search($query) {
          $embedding = app(VectorStoreInterface::class)->embed($query);
          $results = app(VectorStoreInterface::class)->search($embedding);
          return $this->mapResults($results);
      }
  }
  

• Register in config/scout.php:

  'engine' => \App\Scout\Engines\VektorEngine::class,
  

3. Batch Operations

• Pattern: Use Laravel queues for async embedding:

  // app/Jobs/EmbedDocuments.php
  use Symfony\Component\AI\Store\VectorStoreInterface;
  
  class EmbedDocuments implements ShouldQueue {
      public function handle() {
          $documents = Document::all();
          $store = app(VectorStoreInterface::class);
          foreach ($documents as $doc) {
              $store->upsert('doc_'.$doc->id, $this->embed($doc->content), $doc->toArray());
          }
      }
  }
  

• Dispatch via Laravel’s dispatch() or dispatchSync().

4. RAG Pipeline

• Pattern: Retrieve embeddings for LLM prompts:

  $query = "Explain Laravel AI";
  $embedding = $this->store->embed($query);
  $context = $this->store->search($embedding, limit: 2)->map(fn($r) => $r['payload']['content']);
  $prompt = "Context: ".implode("\n", $context)."\nQuestion: ".$query;
  $response = $llm->complete($prompt);
  

Integration Tips

• Environment Variables: Override Vektor’s DSN in .env:

  VEKTOR_DSN=postgres://user:pass@localhost/vektor_db
  

• Payload Filtering: Use payload metadata for Laravel-specific fields (e.g., created_at):

  $this->store->upsert('doc_1', $vector, [
      'title' => 'Post',
      'user_id' => auth()->id(),
      'published_at' => now()->toDateTimeString(),
  ]);
  

• Error Handling: Wrap store operations in Laravel’s try-catch:

  try {
      $results = $this->store->search($embedding);
  } catch (\RuntimeException $e) {
      Log::error("Vektor search failed: ".$e->getMessage());
      return response()->json(['error' => 'Service unavailable'], 503);
  }
  

Gotchas and Tips

Pitfalls

1. Redis vs. PostgreSQL Conflicts:

   • Issue: Laravel’s predis/predis may conflict with Vektor’s Redis client.
   • Fix: Explicitly configure Vektor’s client in the service provider:

     $client = new \Vektor\Client(new \Predis\Client($dsn));
     return new VektorStore($client, $collection);
     

2. Vector Dimension Mismatch:

   • Issue: Embeddings from different models (e.g., text-embedding-ada-002 vs. all-MiniLM-L6-v2) have incompatible dimensions.
   • Fix: Normalize embeddings before storage:

     $embedding = array_map('floatval', $rawEmbedding); // Ensure float[]
     

3. Laravel Caching Interference:

   • Issue: Redis used by both Laravel (cache/queues) and Vektor may cause memory bloat.
   • Fix: Use separate Redis databases:

     VEKTOR_DSN=redis://127.0.0.1:6379/1  # Database 1 for Vektor
     

4. PostgreSQL pgvector Setup:

   • Issue: Missing pgvector extension or incorrect schema.
   • Fix: Run in your PostgreSQL client:

     CREATE EXTENSION vector;
     CREATE TABLE vectors (id TEXT PRIMARY KEY, embedding vector(1536));
     

5. Symfony Component Collisions:

   • Issue: Laravel’s symfony/http-client may conflict with Symfony AI’s dependencies.
   • Fix: Use Laravel’s Http facade or alias Symfony’s client:

     $this->app->alias(\Symfony\Component\HttpClient\HttpClient::class, \Illuminate\Support\Facades\Http::class);
     

Debugging

• Enable Vektor Logging: Configure in config/services.php:

  'vektor' => [
      'dsn' => env('VEKTOR_DSN'),
      'debug' => env('APP_DEBUG', false), // Enable Vektor logs in debug mode
  ],
  

• Laravel Debugbar: Add Vektor metrics via a custom tab:

  // app/Providers/AppServiceProvider.php
  use Barryvdh\Debugbar\Debugbar;
  
  public function boot() {
      Debugbar::addCollector(function() {
          return [
              'Vektor' => [
                  'Collections' => app(\Vektor\Client::class)->listCollections(),
              ],
          ];
      });
  }
  

Extension Points

1. Custom Payload Serialization: Override how Laravel models are serialized to Vektor payloads:

   // app/Services/VektorPayloadSerializer.php
   class VektorPayloadSerializer {
       public function serialize($model) {
           return array_merge(
               $model->toArray(),
               ['serialized_at' => now()->toIso8601String()]
           );
       }
   }
   

   Inject into the store:

   $store = new VektorStore($client, $collection, $serializer);
   

2. Laravel Facade: Create a facade for cleaner syntax:

   // app/Facades/Vektor.php
   use Illuminate\Support\Facades\Facade;
   
   class V