Ai Meilisearch Store Laravel Package

Getting Started

Minimal Setup

1. Install Dependencies:

   composer require symfony/ai-meilisearch-store meilisearch/meilisearch-php
   

2. Configure Meilisearch Client:

   // config/meilisearch.php
   return [
       'host' => env('MEILISEARCH_HOST', 'http://localhost:7700'),
       'api_key' => env('MEILISEARCH_API_KEY', null),
       'index_name' => 'your_vector_index',
   ];
   

3. Bind to Laravel Container (via a service provider):

   // app/Providers/MeilisearchServiceProvider.php
   use Symfony\Component\Ai\Store\MeilisearchStore;
   use Meilisearch\Client;
   
   public function register()
   {
       $this->app->singleton(MeilisearchStore::class, function ($app) {
           $client = new Client(config('meilisearch.host'), config('meilisearch.api_key'));
           return new MeilisearchStore($client, config('meilisearch.index_name'));
       });
   }
   

4. First Use Case:

   // Add embeddings
   $store = app(MeilisearchStore::class);
   $store->add([0.1, 0.2, ...], ['id' => 1, 'content' => 'Sample text']);
   
   // Hybrid search
   $results = $store->search('query', [0.1, 0.2, ...], ['semanticRatio' => 0.5]);
   

Key Files to Review

• src/MeilisearchStore.php: Core implementation.
• tests/: Usage examples and edge cases.
• Meilisearch PHP SDK Docs: Underlying API details.

Implementation Patterns

Core Workflows

1. Vector Indexing

// Batch add embeddings (e.g., from a model)
$embeddings = Model::chunk(100, function ($chunk) use ($store) {
    $documents = $chunk->map(fn ($model) => [
        'embedding' => $model->embedding->toArray(),
        'metadata' => $model->toArray(),
    ]);
    $store->add($documents);
});

2. Hybrid Search

// Combine keyword and vector search
$results = $store->search(
    query: 'user manual',
    vector: $queryEmbedding,
    options: [
        'semanticRatio' => 0.7, // 70% vector, 30% keyword
        'filter' => ['category' => 'docs'],
    ]
);

3. Filtering and Metadata

// Filter by metadata (e.g., user ID)
$results = $store->search(
    query: null, // Pure vector search
    vector: $embedding,
    options: ['filter' => ['user_id' => auth()->id()]]
);

4. Removal

// Remove by ID or vector
$store->remove(['id' => 123]); // By metadata
// OR
$store->remove([], ['embedding' => $vector]); // By vector (advanced)

Integration Tips

Laravel-Specific Patterns

• Service Layer:

  // app/Services/AIService.php
  class AIService {
      public function __construct(
          private MeilisearchStore $store,
          private EmbeddingService $embeddingService
      ) {}
  
      public function semanticSearch(string $query, int $userId): array
      {
          $embedding = $this->embeddingService->create($query);
          return $this->store->search(
              query: $query,
              vector: $embedding,
              options: ['filter' => ['user_id' => $userId]]
          );
      }
  }
  

• Jobs for Async Operations:

  // app/Jobs/IndexEmbeddings.php
  class IndexEmbeddings implements ShouldQueue {
      public function handle() {
          $store = app(MeilisearchStore::class);
          $store->add($this->embeddings, $this->metadata);
      }
  }
  

Meilisearch-Specific Optimizations

• Index Configuration:

  // Configure index via Meilisearch PHP SDK
  $client->index('your_index')->updateSettings([
      'filterableAttributes' => ['user_id', 'category'],
      'rankingRules' => ['semanticScore', 'typo', 'words'],
  ]);
  

• Caching:

  // Cache frequent queries (e.g., Redis)
  $cacheKey = "meilisearch:{$query}:".implode(',', $vector);
  return Cache::remember($cacheKey, now()->addMinutes(5), function () use ($store, $query, $vector) {
      return $store->search($query, $vector);
  });
  

Testing

// PHPUnit test example
public function testHybridSearch()
{
    $store = new MeilisearchStore($this->client, 'test_index');
    $store->add([0.1, 0.2], ['id' => 1]);

    $results = $store->search('test', [0.1, 0.2], ['semanticRatio' => 1]);
    $this->assertCount(1, $results);
}

Gotchas and Tips

Pitfalls

1. Vector Dimension Mismatch:

   • Error: Dimension mismatch when adding vectors.
   • Fix: Ensure all vectors in the index have the same dimension (e.g., 768 for text-embedding-ada-002). Validate during insertion:

     $store->add($embedding, $metadata, ['validate' => true]);
     

2. Meilisearch API Rate Limits:

   • Error: 429 Too Many Requests.
   • Fix: Implement exponential backoff in your client:

     use Symfony\Component\Ai\Store\MeilisearchStore;
     use GuzzleHttp\Exception\RequestException;
     
     try {
         $results = $store->search(...);
     } catch (RequestException $e) {
         if ($e->getCode() === 429) {
             sleep(2); // Retry after delay
             return $store->search(...);
         }
         throw $e;
     }
     

3. Hybrid Search SemanticRatio:

   • Gotcha: semanticRatio > 0.5 may ignore keyword relevance.
   • Tip: Start with 0.3 (30% vector, 70% keyword) and tune based on A/B tests.

4. Filter Syntax:

   • Error: Invalid filter (e.g., user_id:1 vs. user_id=1).
   • Fix: Use Meilisearch’s filter syntax:

     $options = ['filter' => ['user_id = 1', 'category IN (docs, guides)']];
     

5. Laravel Container Conflicts:

   • Error: Class not found if Symfony’s StoreInterface isn’t bound.
   • Fix: Explicitly bind interfaces:

     $this->app->bind(\Symfony\Component\Ai\Store\StoreInterface::class, MeilisearchStore::class);
     

Debugging Tips

• Enable Meilisearch Debugging:

  $client = new Client(config('meilisearch.host'), config('meilisearch.api_key'), [
      'timeout' => 10,
      'debug' => true, // Logs raw API requests
  ]);
  

• Inspect Index:

  curl -X GET http://localhost:7700/indexes/your_index/docs | jq
  

• Laravel Logs:

  \Log::debug('Meilisearch query', [
      'query' => $query,
      'vector' => $vector,
      'options' => $options,
      'results' => $results,
  ]);
  

Extension Points

1. Custom Query Builder:

   // Extend MeilisearchStore to add Laravel-specific methods
   class LaravelMeilisearchStore extends MeilisearchStore {
       public function searchByUserId(string $query, array $vector, int $userId): array
       {
           return $this->search($query, $vector, [
               'filter' => ['user_id = ' . $userId],
           ]);
       }
   }
   

2. Event Dispatching:

   // Dispatch events for vector operations
   event(new VectorAdded($embedding, $metadata));
   

3. Fallback to Keyword Search:

   public function fallbackSearch(string $query): array
   {
       return $this->search($query