Ai Mongo Db Store Laravel Package

Getting Started

Minimal Setup

1. Install the Package:

   composer require symfony/ai-mongo-db-store
   

   Ensure mongodb/mongodb (v2.0+) is installed as a dependency.

2. Configure MongoDB Atlas:

   • Enable Atlas Vector Search on your collection via the Atlas UI.
   • Create a vector index with your embedding dimensions (e.g., 768 for text-embedding-ada-002):

     db.your_collection.createIndex({
       "vector": "vectorSearch",
       "dimensions": 768,
       "similarity": "cosine",
       "name": "vector_index"
     });
     

3. Basic Usage in Laravel:

   use Symfony\AI\Store\MongoDbStore;
   use Symfony\AI\Store\VectorSearchOptions;
   
   // In a service or controller
   $client = new \MongoDB\Client(env('MONGODB_ATLAS_URI'));
   $store = new MongoDbStore(
       $client,
       'your_database',
       'your_collection',
       new VectorSearchOptions(768, 'cosine') // dimensions, similarity metric
   );
   
   // Insert an embedding
   $store->insert([
       'id' => 'doc_123',
       'vector' => [0.1, 0.2, ..., 0.768], // Your embedding array
       'metadata' => ['title' => 'Example', 'category' => 'tech']
   ]);
   
   // Query nearest neighbors
   $results = $store->findNearest(
       [0.5, 0.6, ..., 0.768], // Query vector
       5,                     // Limit
       0.8                    // Threshold (optional)
   );
   

4. Laravel Service Provider: Bind the store to the container for dependency injection:

   // app/Providers/AppServiceProvider.php
   public function register()
   {
       $this->app->singleton(\Symfony\AI\Store\StoreInterface::class, function ($app) {
           $client = new \MongoDB\Client(env('MONGODB_ATLAS_URI'));
           return new MongoDbStore(
               $client,
               env('MONGODB_COLLECTION_DB'),
               env('MONGODB_COLLECTION_NAME'),
               new VectorSearchOptions(768, 'cosine')
           );
       });
   }
   

5. Environment Variables: Add to .env:

   MONGODB_ATLAS_URI=mongodb+srv://user:pass@cluster.mongodb.net/...
   MONGODB_COLLECTION_DB=your_database
   MONGODB_COLLECTION_NAME=your_collection
   

First Use Case: Semantic Search

1. Generate Embeddings: Use a model like text-embedding-ada-002 to create embeddings for your documents (e.g., via Laravel HTTP client or a local service).

2. Store Embeddings:

   $store->insert([
       'id' => 'doc_456',
       'vector' => $embeddingArray,
       'metadata' => ['content' => 'Your document text...']
   ]);
   

3. Query:

   $queryEmbedding = getEmbedding("user search query");
   $results = $store->findNearest($queryEmbedding, 3);
   

4. Display Results:

   foreach ($results as $result) {
       echo $result['metadata']['content']; // Render the matched document
   }
   

Implementation Patterns

Workflows

1. RAG Pipeline Integration

• Retrieve: Use findNearest to fetch relevant documents for an LLM prompt.
• Augment: Combine results with the prompt before sending to the LLM.
• Example:

  $relevantDocs = $store->findNearest($queryEmbedding, 5);
  $context = implode("\n\n", array_map(fn($doc) => $doc['metadata']['content'], $relevantDocs));
  $prompt = "Answer the question based on this context: $context\n\nQuestion: $userQuery";
  $llmResponse = $openAI->chat($prompt);
  

2. Hybrid Search (Vector + Metadata)

• Post-Process Results: Filter findNearest results in PHP using metadata (e.g., category, date).
• Example:

  $rawResults = $store->findNearest($queryEmbedding, 10);
  $filteredResults = array_filter($rawResults, function ($doc) {
      return $doc['metadata']['category'] === 'tech' &&
             strtotime($doc['metadata']['date']) > strtotime('-1 year');
  });
  

3. Batch Operations

• Bulk Insert: Use MongoDB’s bulk write API for efficiency:

  $bulk = new \MongoDB\Driver\BulkWrite;
  foreach ($embeddings as $embedding) {
      $bulk->insert([
          'id' => $embedding['id'],
          'vector' => $embedding['vector'],
          'metadata' => $embedding['metadata']
      ]);
  }
  $client->selectCollection('your_database', 'your_collection')->bulkWrite($bulk->getOperations());
  

4. Dynamic Thresholds

• Adjust findNearest thresholds based on use case (e.g., stricter for high-precision tasks).
• Example:

  $threshold = request()->input('strict') ? 0.9 : 0.7;
  $results = $store->findNearest($queryEmbedding, 5, $threshold);
  

Integration Tips

Laravel-Specific Patterns

1. Queue Jobs for Async Operations:

   • Offload embedding generation/storage to queues:

     // Dispatch a job
     StoreEmbeddingJob::dispatch($documentId, $embeddingArray);
     
     // Job class
     public function handle()
     {
         $store = app(\Symfony\AI\Store\StoreInterface::class);
         $store->insert([...]);
     }
     

2. Caching Layer:

   • Cache frequent queries (e.g., popular search terms) in Laravel’s cache:

     $cacheKey = "search:{$userQuery}";
     $results = cache()->remember($cacheKey, now()->addHours(1), function () use ($queryEmbedding) {
         return $store->findNearest($queryEmbedding, 5);
     });
     

3. Event-Driven Updates:

   • Trigger embedding updates via Laravel events (e.g., after document creation):

     // In a service
     event(new DocumentCreated($document));
     // Listener
     public function handle(DocumentCreated $event)
     {
         $embedding = generateEmbedding($event->document->content);
         $store->insert([...]);
     }
     

4. API Resource Transformation:

   • Shape results for API responses using Laravel’s Resource classes:

     class SearchResultResource extends JsonResource
     {
         public function toArray($request)
         {
             return [
                 'id' => $this->id,
                 'content' => $this->metadata['content'],
                 'score' => $this->score,
             ];
         }
     }
     

MongoDB Atlas Optimization

1. Index Management:

   • Monitor index usage in Atlas UI and rebuild if performance degrades.
   • Use explain() to analyze query plans:

     $collection->explain('executionStats')->find([...]);
     

2. Connection Pooling:

   • Configure MongoDB client pooling in Laravel:

     $client = new \MongoDB\Client($uri, [
         'pool' => [
             'maxSize' => 50,
             'minSize' => 10,
         ],
     ]);
     

3. Atlas Search Integration:

   • Combine with Atlas Search for full-text + vector hybrid queries:

     // Atlas Search pipeline stage
     {
       $search: {
         index: "your_search_index",
         text: { query: "user query", path: "metadata.content" }
       }
     }
     

Gotchas and Tips

Pitfalls

1. Vector Dimensions Mismatch:

   • Issue: Inserting vectors with incorrect dimensions (e.g., 768 vs. 384) will fail silently or return incorrect results.
   • Fix: Validate dimensions before insertion:

     if (count($vector) !== 768) {
         throw new \InvalidArgumentException("Vector must have 768 dimensions.");
     }
     

2. Atlas Vector Search Beta Limitations:

   • Issue: Atlas Vector Search is in beta; some features (e.g., dynamic dimensions, advanced filtering) may be unsupported.
   • Fix: Check Atlas release notes and test