Ai Postgres Store Laravel Package

Getting Started

Minimal Steps

1. Install Dependencies Add to composer.json:

   {
     "require": {
       "symfony/ai": "^0.8",
       "symfony/ai-postgres-store": "^0.8",
       "doctrine/dbal": "^3.6"
     }
   }
   

   Run composer install.

2. Enable pgvector in PostgreSQL Execute in your PostgreSQL client:

   CREATE EXTENSION vector;
   

3. Configure Laravel Add to config/database.php under your PostgreSQL connection:

   'postgres' => [
       'schema' => 'public',
       'pgvector' => [
           'dimensions' => 1536, // Adjust based on your embedding model (e.g., OpenAI's ada-002)
       ],
   ],
   

4. First Use Case: Basic Vector Storage Create a service to interact with the store:

   use Symfony\Component\AI\Store\PostgresStore;
   use Doctrine\DBAL\Connection;
   
   class VectorStoreService {
       public function __construct(private PostgresStore $store) {}
   
       public function addEmbedding(array $embedding, array $metadata) {
           return $this->store->add($embedding, $metadata);
       }
   
       public function findNearest(array $embedding, int $limit = 5) {
           return $this->store->nearest($embedding, $limit);
       }
   }
   

   Bind the service in a Laravel provider:

   $this->app->singleton(VectorStoreService::class, function ($app) {
       $connection = $app->make(Connection::class);
       return new VectorStoreService(
           new PostgresStore($connection->getWrappedConnection(), 'public')
       );
   });
   

5. Create a Migration for Vector Table

   Schema::connection('pgsql')->create('vector_embeddings', function (Blueprint $table) {
       $table->id();
       $table->vector('embedding', 1536); // Adjust dimensions
       $table->json('metadata')->nullable();
       $table->timestamps();
   });
   

6. Run Migrations

   php artisan migrate
   

Where to Look First

• Symfony AI Store Interface: Symfony AI Docs
• pgvector Documentation: pgvector GitHub
• Laravel-Symfony Integration: Spatie Laravel Symfony Bridge

Implementation Patterns

Usage Patterns

1. Basic CRUD Operations

// Add a vector embedding
$store->add([0.1, 0.2, ...], ['product_id' => 123, 'category' => 'electronics']);

// Find nearest neighbors
$neighbors = $store->nearest([0.15, 0.25, ...], 5);

// Remove a vector by ID
$store->remove(1);

2. Hybrid Search (Vector + Full-Text)

// Combine vector similarity with metadata filtering
$results = $store->search(
    [0.1, 0.2, ...],
    10,
    [
        'filter' => [
            'category' => 'electronics',
            'price_gt' => 100,
        ],
        'text_search' => 'wireless headphones' // PostgreSQL full-text search
    ]
);

3. Batch Operations

// Add multiple embeddings in a batch
$batch = [
    ['embedding' => [0.1, 0.2, ...], 'metadata' => ['id' => 1]],
    ['embedding' => [0.3, 0.4, ...], 'metadata' => ['id' => 2]],
];
$store->addBatch($batch);

4. Integration with Laravel Eloquent

Use a custom accessor to fetch embeddings for Eloquent models:

// In your Product model
public function getEmbeddingAttribute() {
    return $this->vectorEmbeddings()->value('embedding');
}

public function vectorEmbeddings() {
    return $this->hasOne(VectorEmbedding::class);
}

Workflows

Semantic Search Workflow

1. Generate Embeddings: Use a model (e.g., OpenAI, Sentence Transformers) to generate embeddings for search queries and documents.
2. Store Embeddings: Add embeddings to the PostgreSQL store with metadata (e.g., document ID, category).
3. Query Embeddings: For a user query, generate its embedding and find nearest neighbors in the store.
4. Return Results: Combine results with metadata (e.g., product names, descriptions) for display.

Recommendation Engine Workflow

1. User Interaction Tracking: Store user interactions (e.g., clicks, purchases) as embeddings with user IDs.
2. Generate Recommendations: For a user, find nearest neighbors to their interaction embeddings.
3. Filter by Context: Apply metadata filters (e.g., "only recommend products in category X").

Integration Tips

Laravel Service Container

Wrap the Symfony store in a Laravel service for easier testing and dependency injection:

class VectorStoreService {
    public function __construct(private PostgresStore $store) {}

    public function search(array $embedding, int $limit, array $filters = []) {
        return $this->store->search($embedding, $limit, $filters);
    }
}

Doctrine DBAL for Raw Queries

For complex queries, use Doctrine DBAL to bypass Laravel’s Eloquent limitations:

use Doctrine\DBAL\Connection;

class VectorQueryBuilder {
    public function __construct(private Connection $connection) {}

    public function findSimilarProducts(array $embedding, int $limit) {
        $sql = "
            SELECT *, embedding <=> :embedding AS distance
            FROM vector_embeddings
            WHERE embedding <=> :embedding < 0.5
            ORDER BY distance
            LIMIT :limit
        ";
        return $this->connection->fetchAllAssociative($sql, [
            'embedding' => $embedding,
            'limit' => $limit,
        ]);
    }
}

Caching Frequent Queries

Cache results of expensive vector searches using Laravel’s cache:

public function cachedNearest(array $embedding, int $limit, string $cacheKey) {
    return Cache::remember($cacheKey, now()->addHours(1), function () use ($embedding, $limit) {
        return $this->store->nearest($embedding, $limit);
    });
}

Event Listeners for Vector Updates

Trigger actions when vectors are added or removed:

// In EventServiceProvider
protected $listen = [
    VectorAdded::class => [
        UpdateSearchIndex::class,
    ],
    VectorRemoved::class => [
        CleanupRecommendations::class,
    ],
];

Gotchas and Tips

Pitfalls

1. pgvector Extension Not Enabled

• Symptom: Queries fail with ERROR: function vector(...) not found.
• Fix: Ensure CREATE EXTENSION vector; is run in PostgreSQL. Verify with:

  SELECT * FROM pg_extension WHERE extname = 'vector';
  

2. Dimension Mismatch

• Symptom: ERROR: dimension mismatch when adding vectors.
• Fix: Ensure all vectors in your table have the same dimensions. Configure the dimensions option in your Laravel config to match your embeddings (e.g., 1536 for OpenAI’s text-embedding-ada-002).

3. Laravel Eloquent Limitations

• Symptom: Eloquent cannot handle vector columns natively.
• Fix: Use raw SQL or Doctrine DBAL for queries involving vector columns. Example:

  $results = DB::connection('pgsql')->select(
      "SELECT *, embedding <=> :embedding AS distance FROM vector_embeddings ORDER BY distance LIMIT 10",
      ['embedding' => $yourEmbedding]
  );
  

4. Symfony Dependency Conflicts

• Symptom: Composer conflicts due to Symfony version requirements.
• Fix: Pin Symfony dependencies in composer.json:

  "require": {
      "symfony/ai": "^0.8",
      "symfony/ai-postgres-store": "^0.8",
      "symfony/http-client": "^6.0",
      "symfony/options-resolver": "^6.0"
  }
  

5. Performance Degradation with Large Datasets

• Symptom: Queries slow down as the number of vectors grows beyond 1M.
• Fix:
  • Partition the table by metadata (e.g., category).
  • Use halfvec type for 8-bit precision (halves memory usage