Weave Code
Weaver
Helps Laravel developers discover, compare, and choose open-source packages. See popularity, security, maintainers, and scores at a glance to make better decisions.
Feedback
Share your thoughts, report bugs, or suggest improvements.
Subject
Message

Ai Maria Db Store Laravel Package

symfony/ai-maria-db-store

MariaDB vector store integration for Symfony AI Store, enabling vector search and RAG with MariaDB's native VECTOR type, VECTOR INDEX, and distance functions. Requires MariaDB 11.7 or later.

View on GitHub
Deep Wiki
Context7

Getting Started

Minimal Setup for Laravel

  1. Install the Package:

    composer require symfony/ai-maria-db-store:^0.15
    

    Ensure your composer.json includes Symfony’s AI components if not already present, and pin the whole family to ^0.15:

    "require": {
        "symfony/ai": "^0.15",
        "symfony/dependency-injection": "^7.0"
    }
    
  2. Configure MariaDB:

    • Upgrade to MariaDB 11.8+ (check with mariadb --version). 0.15 raises the floor from 11.7 to 11.8 — Distance handling depends on it.
    • Create a table with a VECTOR column and let the store build the index, or create it yourself with the same distance metric you configure in PHP:
      CREATE TABLE ai_embeddings (
          id INT AUTO_INCREMENT PRIMARY KEY,
          content TEXT,
          metadata JSON,
          embedding VECTOR(1536),  -- Adjust dimensions to your embedding size
          INDEX vec_idx USING HNSW(embedding) WITH (distance_type = 'COSINE')
      ) ENGINE=InnoDB;
      
    • Update Laravel’s .env:
      DB_MARIADB_CONNECTION=mariadb
      DB_MARIADB_URL="mysql://user:pass@host/db?serverVersion=11.8"
      
  3. Register the Store: Add a service provider (e.g., App\Providers\MariaDbStoreServiceProvider):

    use Symfony\Component\AI\Store\AiStoreInterface;
    use Symfony\AI\MariaDbStore\MariaDbStore;
    
    public function register()
    {
        $this->app->singleton(AiStoreInterface::class, function ($app) {
            return new MariaDbStore(
                $app['db']->connection('mariadb')->getPdo(),
                [
                    'table' => 'ai_embeddings',
                    'vector_column' => 'embedding',
                    'distance' => 'cosine',   // 0.15 creates the index with this metric
                    'dimensions' => 1536,
                ]
            );
        });
    }
    
  4. First Use Case: Insert and query embeddings in a Laravel controller:

    use Symfony\Component\AI\Store\AiStoreInterface;
    
    public function storeEmbedding(AiStoreInterface $store)
    {
        $embedding = [0.1, 0.2, ..., 0.1]; // Your 1536-dim vector
        $store->insert([
            'content' => 'Sample text',
            'metadata' => ['category' => 'tech'],
            'embedding' => $embedding,
        ]);
    
        // Query with similarity (vector is parsed once per query in 0.15)
        $results = $store->query($embedding, [
            'limit' => 5,
            'filter' => ['category' => 'tech'],
        ]);
    
        // Cheap size checks without fetching anything
        $total   = $store->count();
        $techCnt = $store->count(['category' => 'tech']);
    
        // Results are VectorDocumentInterface instances
        foreach ($results as $document) {
            // $document->getId(), $document->getContent(), $document->getMetadata()
        }
    }
    

Implementation Patterns

Core Workflows

  1. CRUD Operations:

    • Insert: Use insert() for single records or insertMany() for batches:
      $store->insertMany([
          ['content' => 'Doc 1', 'embedding' => $vec1, 'metadata' => [...]],
          ['content' => 'Doc 2', 'embedding' => $vec2, 'metadata' => [...]],
      ]);
      
    • Count: count() accepts the same filter shape as query(), so you can build paginators, dashboards, and “is the index populated?” health checks without transferring vectors:
      $paginator = new LengthAwarePaginator(
          items: $store->query($vector, ['limit' => $perPage, 'offset' => $offset]),
          total: $store->count($filters),
          perPage: $perPage,
          currentPage: $page,
      );
      
    • Update: Replace records via insert() (no native update; use remove() + insert()).
    • Remove: Delete by ID or filter:
      $store->remove(['id' => 1]); // By ID
      $store->remove(['category' => 'deprecated']); // By metadata filter
      
  2. Working with Documents (VectorDocumentInterface):

    • Query results are typed against Symfony\Component\AI\Store\VectorDocumentInterface. Type-hint the interface in your own code rather than assuming a concrete class or a plain array shape:
      use Symfony\Component\AI\Store\VectorDocumentInterface;
      
      $documents = collect($store->query($vector, ['limit' => 10]))
          ->map(fn (VectorDocumentInterface $doc) => new SearchHit(
              id: $doc->getId(),
              text: $doc->getContent(),
              meta: $doc->getMetadata(),
          ));
      
    • Do not pass raw arrays to code expecting documents, and don’t hand-roll a result object — if you need a custom document (e.g. an Eloquent-backed EmbeddingDocument), implement VectorDocumentInterface so it stays compatible with the store.
    • Map at the boundary (repository/action) into your app’s DTOs so an interface change in Symfony doesn’t ripple through your views.
  3. Hybrid Queries: Combine vector similarity with SQL filters:

    $results = $store->query($queryVector, [
        'limit' => 10,
        'filter' => [
            'category' => 'tech',
            'created_at' => ['>', '2023-01-01'],
        ],
    ]);
    
    • Supported Filters: Use MariaDB’s WHERE syntax (e.g., JSON_EXTRACT(metadata, '$.priority') > 5).
    • 0.15 parses the query vector once per query instead of re-serializing per clause — wide filters on high-dimensional vectors get noticeably cheaper, but it is still a per-query cost, not free.
  4. Batch Processing:

    • Bulk Insert: Use insertMany() for efficiency (test with 1K+ records).
    • Parallel Queries: Offload heavy queries to queues (e.g., Laravel Horizon):
      dispatch(new ProcessEmbeddings($store, $batch));
      
  5. Schema Management:

    • Distance is baked into the index: since 0.15 the store creates the vector index with the distance metric from your config. Decide distance (cosine/l2) up front — changing it later means dropping and recreating the index, and a pre-existing hand-made index with a different metric won’t match.
    • Dynamic Dimensions: Recreate the table if dimensions change (no ALTER TABLE support for VECTOR).
    • Indexing: You can still add/drop indexes manually for bulk loads:
      ALTER TABLE ai_embeddings ADD INDEX vec_idx USING HNSW(embedding)
        WITH (distance_type = 'COSINE');  -- must match the store's `distance`
      

Integration Tips

  • Laravel-Specific:

    • Use Repositories: Isolate store logic in a repository to avoid mixing Eloquent/PDO:
      class EmbeddingRepository {
          public function __construct(private AiStoreInterface $store) {}
      
          public function findSimilar($vector, array $filters = []) {
              return $this->store->query($vector, ['filter' => $filters]);
          }
      
          public function total(array $filters = []): int {
              return $this->store->count($filters);
          }
      }
      
    • Artisan Command: A ai:stats command using count() is a cheap way to verify a deploy/indexing job actually wrote rows.
    • Events: Trigger events for critical operations (e.g., EmbeddingStored):
      event(new EmbeddingStored($record));
      
  • Symfony Abstraction:

    • Extend AiStoreInterface for custom methods:
      interface CustomStoreInterface extends AiStoreInterface {
          public function getByMetadata(array $filters);
      }
      
    • Any decorator/cache wrapper around the store must forward count() and return VectorDocumentInterface instances — a missed forward means silent BadMethodCallExceptions only in production paths.
  • Testing:

    • Mock the store in unit tests, and stub the methods your code path touches:
      $store = $this->createMock(AiStoreInterface::class);
      $store->method('query')->willReturn([/* VectorDocumentInterface instances */]);
      $store->method('count')->willReturn(42);
      
    • Run the suite against MariaDB 11.8 (a container, not SQLite) now that the driver depends on 11.8 semantics.

Gotchas and Tips

Pitfalls

  1. MariaDB Version:

    • Error: Unknown column type 'VECTOR' → You’re on MariaDB <11.7.
    • Error after upgrading to 0.15: distance-related failures on 11.7 → 0.15 requires 11.8+ for Distance. Pin serverVersion=11.8 in the DSN so the platform detection matches the server.
    • Fix: Upgrade to 11.8 or use a compatible alternative (e.g., pgvector).
  2. Distance Functions:

    • Error: Unknown system variable 'distance_type' → HNSW index misconfiguration.
    • Fix: Use COSINE or L2 (Euclidean) explicitly, and make sure it matches the distance option passed to MariaDbStore:
      INDEX vec_idx USING HNSW(embedding) WITH (distance_type = 'COSINE')
      
    • Since 0.15 the store generates the index with its own configured distance, so an index created earlier by hand with a different metric will silently change result ordering expectations.
  3. Vector Dimensions:

    • Error: Data too long for column 'embedding' → Mismatched dimensions.
    • Fix: Ensure dimensions in config match the VECTOR(N) column.
  4. Hybrid Query Limits:

    • Error: SQLSTATE[HY000]: General error: 1118 → Complex filters may fail.
    • Fix: Simplify filters or use raw SQL:
      $store->query($vector, ['filter' => ['raw' => 'category = ? AND priority > ?', ['tech', 5]]]);
      
  5. Laravel-Symfony Conflicts:

    • Error: Class 'Symfony\Component\AI\Store\AiStoreInterface' not found.
    • Fix: Ensure symfony/ai is installed and autoloaded. Pin ^0.15 across the whole family — mixing a 0.14 store with a 0.15 symfony/ai (or vice versa) is the most likely source of interface-mismatch fatals after this release.
  6. Upgrading 0.14 → 0.15:

    • No database migration is required, but the minimum server version moved to 11.8 and index creation now uses the store’s configured distance. Verify distance in config matches the metric your existing vec_idx was built with; recreate the index if not.
    • Nothing in the public API changed — query() still returns VectorDocumentInterface instances and count() is unchanged. The “parse the query vector once per query” change is internal; only custom subclasses overriding vector placeholder/parsing logic need a look.
    • Keep the ^0.15 bump applied to symfony/ai, symfony/ai-maria-db-store, and any other AI sub-packages in one PR to avoid a split-brain install.

Debugging Tips

  • Enable MariaDB Logging:
    [mysqld]
    general_log = 1
    general_log_file = /var/log/mysql/mariadb-query.log
    
  • Query Inspection: Use DB::enableQueryLog() to capture raw SQL:
    DB::enableQueryLog();
    $store->query($vector);
    dd(DB::getQueryLog());
    
  • Count Sanity Check: when count() returns 0 but query() returns hits, your filter array is being interpreted differently by each path — verify against the raw SQL from the query log.
  • Index Sanity Check: SHOW INDEX FROM ai_embeddings / SHOW CREATE TABLE to confirm the HNSW index exists and carries the metric you configured.

Performance Quirks

  1. Indexing Overhead:

    • HNSW indexes improve performance but slow down inserts. Test with:
      -- Disable index temporarily for bulk inserts
      ALTER TABLE ai_embeddings DROP INDEX vec_idx;
      -- Insert data
      ALTER TABLE ai_embeddings ADD INDEX vec_idx USING HNSW(embedding)
        WITH (distance_type = 'COSINE');
      
    • Note that a store bootstrapped after the drop may try to create the index itself — re-add it with the store’s metric.
  2. Batch Size:

    • Optimal insertMany() batch size: 500–2000 records (benchmark with EXPLAIN).
  3. Distance Metrics:

    • Cosine is faster for high-dimensional vectors (>512D) than Euclidean.
  4. count() Cost:

    • Unfiltered count() on InnoDB scans the index; filtered count() may scan the table. Use it for periodic stats, not for real-time UI badges on hot paths.

Extension Points

  1. Custom Distance Functions: Override the store’s getDistanceSql() method (keep the default distance mapping intact so the index and the query agree):

    class CustomMariaDbStore extends MariaDbStore {
        protected function getDistanceSql(string $column, string $distance): string
        {
            return match ($distance) {
                'custom' => "1 - (($column) * $this->getVectorPlaceholder())",
                default => parent::getDistanceSql($column, $distance),
            };
        }
    }
    
  2. Custom Documents: Implement VectorDocumentInterface for Eloquent-backed results:

    use Symfony\Component\AI\Store\VectorDocumentInterface;
    
    class EmbeddingDocument implements VectorDocumentInterface
    {
        public function __construct(
            private string $id,
            private string $content,
            private array $metadata = [],
        ) {}
    
        public function getId(): string { return $this->id; }
        public function getContent(): string { return $this->content; }
        public function getMetadata(): array { return $this->metadata; }
    }
    
  3. Metadata Serialization: Extend to handle custom metadata formats (e.g., arrays):

    protected function serializeMetadata(array $metadata): string
    {
        return json_encode($metadata, JSON_THROW_ON_ERROR);
    }
    
  4. Async Operations: Use Laravel Queues for long-running queries:

    class ProcessVectorQuery implements ShouldQueue {
        public function handle(AiStoreInterface $store): void {
            $results = $store->query($this->vector, ['limit' => $this->limit]);
            // handle VectorDocumentInterface results
        }
    }
    
Weaver

How can I help you explore Laravel packages today?

Conversation history is not saved when not logged in.
Prompt
Add packages to context
No packages found.
socialdept/atp-orm
cboxdk/laravel-assistant-client
socialdept/atp-resolver
socialdept/tenant-domains
tautvydasr/blade-class-names
socialdept/color-scales
maherelgamil/controller
coleus/widgets
coleus/table
coleus/users
coleus/music
coleus/health
coleus/calendar
coleus/notes
coleus/support
coleus/settings
coleus/apps
seampact/laravel
emmyconceptng/laravel-ng-postcode
jjsquady/round-robin