Ai Mongo Db Store Laravel Package

Getting Started

Minimal Setup

1. Install Dependencies:

   composer require symfony/ai-mongo-db-store mongodb/mongodb
   

2. Configure MongoDB Atlas:
   • Enable Vector Search on your collection via Atlas UI or CLI:

     db.embeddings.createIndex({
       "vector": "vectorSearch",
       "dimensions": 768,  // Match your embedding dimension
       "similarity": "cosine"  // or 'euclidean', 'dotProduct'
     });
     

3. Basic Usage in Laravel:

   use Symfony\AI\Store\MongoDbStore;
   use MongoDB\Client;
   
   $client = new Client(env('MONGODB_ATLAS_URI'));
   $store = new MongoDbStore(
       $client->selectDatabase('your_db'),
       'embeddings',
       new \Symfony\AI\Store\MongoDbStore\VectorSearchOptions(768, 'cosine')
   );
   
   // Insert an embedding
   $store->insert('doc_id', [0.1, 0.2, ...]); // 768-dim vector
   
   // Query nearest neighbors
   $results = $store->findNearest('query_vector', 5);
   

First Use Case: Semantic Search

Replace a keyword search with vector-based retrieval:

// Generate embeddings (e.g., with Symfony AI's embedder)
$embedding = $embeddingService->embed("user query");

// Retrieve top 3 semantically similar documents
$similarDocs = $store->findNearest($embedding, 3);

// Use results in your app (e.g., populate search results)
foreach ($similarDocs as $doc) {
    $content = $documentRepository->find($doc['id'])->getContent();
    // ...
}

Implementation Patterns

Workflows

1. Embedding Pipeline:

   // Laravel Service Example
   public function search(string $query, int $limit = 5) {
       $embedding = $this->embeddingService->embed($query);
       $ids = $this->vectorStore->findNearest($embedding, $limit)
           ->map(fn ($item) => $item['id']);
   
       return $this->documentRepository->findByIds($ids);
   }
   

2. Hybrid Search (Vector + Metadata): Combine with MongoDB’s $match for filtering:

   $query = [
       '$vectorSearch' => [
           'queryVector' => $embedding,
           'path' => 'vector',
           'numCandidates' => 100,
           'limit' => 5,
           'index' => 'vector_index'
       ],
       '$match' => ['category' => 'books', 'publishedAfter' => new \MongoDB\BSON\UTCDateTime('2020-01-01')]
   ];
   $cursor = $this->collection->aggregate([$query]);
   

Integration Tips

• Laravel Service Provider: Bind the store to the container for dependency injection:

  public function register() {
      $this->app->singleton(MongoDbStore::class, function ($app) {
          return new MongoDbStore(
              new Client(env('MONGODB_ATLAS_URI')),
              env('MONGODB_COLLECTION'),
              new VectorSearchOptions(768, 'cosine')
          );
      });
  }
  

• Batch Operations: Use bulk writes for efficiency:

  $bulk = $this->collection->initializeUnorderedBulkOp();
  foreach ($documents as $doc) {
      $bulk->insert([
          'id' => $doc['id'],
          'vector' => $doc['embedding'],
          'metadata' => $doc['metadata']
      ]);
  }
  $bulk->execute();
  

• Caching: Cache frequent queries (e.g., top recommendations) in Redis:

  $cacheKey = "recommendations:{$userId}";
  $recommendations = cache()->remember($cacheKey, now()->addHours(1), function () use ($userId) {
      return $this->vectorStore->findNearest($this->getUserEmbedding($userId), 10);
  });
  

Common Patterns

Gotchas and Tips

Pitfalls

1. Index Mismatch Errors:

   • Issue: Invalid dimensions or similarity metric not supported.
   • Fix: Ensure the VectorSearchOptions match your Atlas index:

     // Atlas index: dimensions=768, similarity=cosine
     $store = new MongoDbStore(..., new VectorSearchOptions(768, 'cosine'));
     

   • Debug: Check Atlas UI for index definition or run:

     db.embeddings.getIndexes();
     

2. Connection Timeouts:

   • Issue: Slow queries or timeouts in development.
   • Fix: Use a local MongoDB instance for testing (disable Atlas Vector Search for faster iterations):

     // config/services.php
     'mongodb.connection' => env('MONGODB_LOCAL_URI', env('MONGODB_ATLAS_URI')),
     

   • Tip: Enable Atlas serverless instance for predictable performance.

3. Vector Dimension Mismatch:

   • Issue: Error: Vector dimension mismatch when inserting.
   • Fix: Validate embedding dimensions before insertion:

     if (count($embedding) !== 768) {
         throw new \InvalidArgumentException("Embedding must be 768-dimensional.");
     }
     

4. Filter Syntax:

   • Issue: MongoDB filter syntax errors (e.g., $gt vs. >).
   • Fix: Use BSON operators or the MongoDB\BSON class:

     $filter = [
         'price' => ['$gt' => 50],
         'category' => 'books'
     ];
     

5. Rate Limiting:

   • Issue: Atlas free tier or low-tier limits queries.
   • Fix: Monitor Atlas operation counts and upgrade if needed.
   • Tip: Cache results aggressively for high-traffic endpoints.

Debugging

• Query Explanation: Use $explain to analyze performance:

  $explanation = $this->collection->aggregate([
      ['$vectorSearch' => [...]],
      ['$limit' => 1]
  ], ['explain' => true]);
  

• Logs: Enable MongoDB logging in php.ini:

  mongodb.log.appendOnly = true
  mongodb.log.path = /var/log/mongodb.log
  

• Symfony Debug: Dump the store’s MongoDB client for connection details:

  \Symfony\Component\Debug\Debug::dump($store->getCollection()->getManager()->getDatabase()->getClient());
  

Configuration Quirks

1. Environment Variables:

   • Store sensitive credentials in .env:

     MONGODB_ATLAS_URI=mongodb+srv://user:pass@cluster.mongodb.net/db?retryWrites=true
     MONGODB_COLLECTION=embeddings
     

   • Tip: Use Laravel’s config/mongodb.php for centralized config:

     'connections' => [
         'atlas' => [
             'uri' => env('MONGODB_ATLAS_URI'),
             'database' => env('MONGODB_DATABASE', 'vector_db'),
             'options' => [
                 'retryWrites' => true,
                 'retryReads' => true,
             ],
         ],
     ],
     

2. Atlas-Specific Settings: