Ai S3Vectors Store Laravel Package

Getting Started

Minimal Steps

1. Install the Package:

   composer require symfony/ai-s3vectors-store
   

2. Configure the Store in config/packages/ai.yaml:

   framework:
       ai:
           stores:
               s3_vectors:
                   type: S3VectorsStore
                   bucket: your-bucket-name
                   region: us-east-1
                   aws:
                       credentials:
                           key: "%env(AWS_ACCESS_KEY_ID)%"
                           secret: "%env(AWS_SECRET_ACCESS_KEY)%"
   

3. Enable S3 Vectors on your bucket (via AWS CLI or Console):

   aws s3api put-bucket-vector --bucket your-bucket-name --region us-east-1
   

4. First Usage (inject the store via Symfony’s DI or manually):

   use Symfony\AI\Store\S3VectorsStore;
   
   $store = new S3VectorsStore('your-bucket-name', 'us-east-1');
   $store->putVectors([
       'vector1' => [0.1, 0.2, 0.3], // Example vector
       'vector2' => [0.4, 0.5, 0.6],
   ]);
   
   $results = $store->queryVectors([0.15, 0.25, 0.35], 5); // Query with a vector, limit 5 results
   

Where to Look First

• Symfony AI Documentation: Symfony AI Stores for store interface details.
• AWS S3 Vectors API: PutVectors and QueryVectors for payload/response formats.
• Package Source: S3VectorsStore class for method signatures and error handling.

First Use Case

Semantic Search for a Blog:

1. Store embeddings of blog posts in S3 Vectors.
2. Query with a user’s search query (converted to a vector) to retrieve top-5 relevant posts.

   $embeddings = $this->generateEmbeddings($userQuery);
   $results = $this->ai->store('s3_vectors')->queryVectors($embeddings, 5);
   

Implementation Patterns

Core Workflows

1. Vector Storage

• Batch Insertion (recommended for performance):

  $store->putVectors([
      'post_1' => [0.1, 0.2, ...], // Vector ID as key
      'post_2' => [0.3, 0.4, ...],
  ]);
  

• Single Insertion (for dynamic vectors):

  $store->putVectors(['dynamic_id' => $vector]);
  

2. Vector Querying

• Approximate Nearest Neighbor (ANN) Search:

  $results = $store->queryVectors($queryVector, 10); // Top 10 matches
  

  • Returns an array of ['id' => 'vector_id', 'distance' => float].
• Exact Match (if needed, treat as a query with a trivial vector).

3. Hybrid Integration with Symfony AI

• Use Symfony’s AiClient to abstract store operations:

  $this->ai->store('s3_vectors')->putVectors($vectors);
  $matches = $this->ai->store('s3_vectors')->queryVectors($query, 5);
  

4. Error Handling

• Wrap operations in try-catch for S3-specific errors:

  try {
      $results = $store->queryVectors($vector);
  } catch (AwsException $e) {
      $this->logger->error('S3 Vectors query failed', ['error' => $e->getMessage()]);
      // Fallback logic (e.g., retry or return cached results)
  }
  

Integration Tips

AWS Configuration

• Credentials: Use Symfony’s %env% or AWS SDK’s default chain (e.g., ~/.aws/credentials).
• Region: Ensure the bucket and region match in both S3 and the store config.
• IAM Permissions: Grant the IAM role/user:

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "s3:PutBucketVector",
                  "s3:GetBucketVector",
                  "s3:PutObject",
                  "s3:GetObject"
              ],
              "Resource": [
                  "arn:aws:s3:::your-bucket-name",
                  "arn:aws:s3:::your-bucket-name/*"
              ]
          }
      ]
  }
  

Vector Format

• Supported Types: Vectors must be floating-point arrays (e.g., [0.1, 0.2, 0.3]).
• Dimensions: S3 Vectors supports up to 10,000 dimensions (check AWS limits).
• IDs: Use strings as vector IDs (e.g., "user_123", "post_456").

Performance Optimization

• Batch Size: Test with batches of 50–100 vectors to balance latency and throughput.
• Parallel Queries: For high-throughput apps, use Symfony’s Messenger or SymfonyConcurrent to parallelize queries.
• Caching: Cache frequent queries (e.g., trending recommendations) in Redis or OPcache.

Testing

• Mock S3: Use Aws\S3\S3Client with a mock bucket for unit tests:

  $mock = new Aws\S3\MockS3Client();
  $store = new S3VectorsStore('mock-bucket', 'us-east-1', $mock);
  

• Load Testing: Simulate production traffic with tools like Artillery or Locust to measure:
  • PutVectors latency (aim for <500ms for batches).
  • QueryVectors throughput (aim for <1s for top-10 results).

Gotchas and Tips

Pitfalls

1. S3 Vectors Not Enabled:

   • Error: InvalidArgumentException or 400 Bad Request when calling PutVectors.
   • Fix: Verify the bucket has S3 Vectors enabled via AWS Console or:

     aws s3api get-bucket-vector --bucket your-bucket-name
     

2. Vector Size Limits:

   • Error: PayloadTooLargeException if vectors exceed 1MB.
   • Fix: Compress vectors (e.g., with gzip) or split into multiple objects.

3. AWS SDK Version Mismatch:

   • Error: ClassNotFoundException for Aws\S3\S3Client.
   • Fix: Ensure aws/aws-sdk-php is installed (^3.0) and compatible with the package.

4. Throttling:

   • Error: ProvisionedThroughputExceededException during high-volume operations.
   • Fix: Implement exponential backoff in retries:

     $client->retryConfig->setMaxRetries(3);
     

5. Vector ID Collisions:

   • Error: Overwriting vectors silently if IDs are reused.
   • Fix: Use UUIDs or composite IDs (e.g., user_{id}_post_{id}).

6. Cold Starts:

   • Issue: First query after inactivity may be slower (~500ms–1s).
   • Fix: Pre-warm the bucket with a dummy query or use a CDN-like cache layer.

Debugging Tips

• Enable AWS SDK Debugging:

  $client = new Aws\S3\S3Client([
      'region' => 'us-east-1',
      'debug' => true,
      'logger' => new Aws\Log\NullLogger(), // Or Psr\Log\LoggerInterface
  ]);
  

• Check S3 Vectors Logs:
  • Use AWS CloudTrail to audit PutVectors/QueryVectors API calls.
• Validate Payloads:
  • Log the exact payload sent to S3 to match against AWS’s API docs.

Configuration Quirks

• Default Region: If omitted, falls back to AWS_REGION env var or us-east-1.
• Credentials: Overrides Symfony’s `%env(AWS