Pinecone Php Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require probots-io/pinecone-php
   

   Add the service provider in config/app.php (if not auto-discovered):

   'providers' => [
       Probots\Pinecone\PineconeServiceProvider::class,
   ],
   

2. Configuration Publish the config file:

   php artisan vendor:publish --provider="Probots\Pinecone\PineconeServiceProvider" --tag="config"
   

   Update .env with your Pinecone API key, environment, and index host (if not using default):

   PINECONE_API_KEY=your_api_key_here
   PINECONE_ENVIRONMENT=your_env_here
   PINECONE_INDEX_HOST=your-custom-host.pinecone.io  # Optional, if not default
   

3. First Use Case: Querying an Index

   use Probots\Pinecone\Facades\Pinecone;
   
   $results = Pinecone::query('your-index-name', 'your-query-vector', [
       'topK' => 5,
       'includeMetadata' => true,
   ]);
   

Implementation Patterns

Core Workflows

1. Vector Operations

   • Upsert Vectors:

     Pinecone::upsert('your-index-name', [
         ['id' => '1', 'values' => [0.1, 0.2, ...], 'metadata' => ['category' => 'test']],
         ['id' => '2', 'values' => [0.3, 0.4, ...], 'metadata' => ['category' => 'demo']],
     ]);
     

   • Query Vectors:

     $query = Pinecone::query('your-index-name', [0.15, 0.25, ...], ['topK' => 3]);
     

2. Index Management

   • Create/Delete Index:

     Pinecone::createIndex('new-index-name', ['dimensions' => 1536, 'metric' => 'cosine']);
     Pinecone::deleteIndex('old-index-name');
     

   • Fetch Index Stats:

     $stats = Pinecone::describeIndex('your-index-name');
     

3. Collections (New in 1.1.0) Pinecone now supports collections (multi-index management). Use the collections() method:

   // List all collections
   $collections = Pinecone::collections();
   
   // Create a collection
   Pinecone::createCollection('my-collection', ['indexes' => ['index1', 'index2']]);
   
   // Add an index to a collection
   Pinecone::addIndexToCollection('my-collection', 'new-index');
   

4. Batch Operations

   • Bulk Upsert/Delete:

     Pinecone::upsert('index-name', $batchData, ['batchSize' => 100]);
     Pinecone::delete('index-name', ['id1', 'id2', ...]);
     

5. Metadata Filtering

   • Filter Queries:

     Pinecone::query('index-name', $vector, [
         'filter' => ['category' => ['$eq' => 'test']],
         'topK' => 5,
     ]);
     

Integration Tips

• Laravel Eloquent: Sync Pinecone with your database using events:

  public function boot()
  {
      YourModel::created(function ($model) {
          Pinecone::upsert('vectors-index', [
              ['id' => $model->id, 'values' => $model->vectorize(), 'metadata' => $model->toArray()]
          ]);
      });
  }
  

• Queue Jobs: Offload heavy operations to queues (e.g., PineconeUpsertJob).
• Caching: Cache frequent queries or index stats in Redis/Memcached.
• PHP Version Support: The package now supports multiple PHP versions (7.4+, 8.0+, 8.1+). No changes required unless using legacy PHP.

Gotchas and Tips

Pitfalls

1. Rate Limiting Pinecone enforces rate limits. Handle Probots\Pinecone\Exceptions\RateLimitExceededException:

   try {
       Pinecone::query(...);
   } catch (RateLimitExceededException $e) {
       sleep($e->getRetryAfter());
       retry();
   }
   

2. Vector Dimensions Mismatched dimensions (e.g., upserting a 768-dim vector into a 1536-dim index) throws InvalidArgumentException. Tip: Validate dimensions before upserting:

   $index = Pinecone::describeIndex('index-name');
   if (count($vector) !== $index['dimensions']) {
       throw new \InvalidArgumentException("Vector dimension mismatch");
   }
   

3. Metadata Schema Pinecone doesn’t enforce schema validation. Use JSON Schema to validate metadata:

   use Justinrainbow\JsonSchema\Validator;
   $validator = new Validator();
   $validator->validate($metadata, $schema);
   

4. ID Collisions Upserting duplicate IDs overwrites existing vectors. Check for conflicts:

   $existing = Pinecone::fetch('index-name', ['id1', 'id2']);
   if ($existing->contains('id1')) {
       // Handle update logic
   }
   

5. Index Host Configuration (New in 1.1.0) If using a custom Pinecone host (e.g., your-custom-host.pinecone.io), ensure it’s set in .env:

   PINECONE_INDEX_HOST=your-custom-host.pinecone.io
   

   Tip: Verify the host format in the Pinecone docs.

Debugging

• Enable Debug Mode:

  Pinecone::setDebug(true); // Logs raw API requests/responses
  

• Common Exceptions:
  • IndexNotFoundException: Verify index name, environment, and host.
  • AuthenticationException: Check PINECONE_API_KEY and permissions.
  • ValidationException: Validate payload structure (e.g., values must be an array of floats).

Extension Points

1. Custom HTTP Client Override the default Guzzle client for logging/monitoring:

   Pinecone::setClient(new CustomGuzzleClient());
   

2. Collections (New in 1.1.0) Extend collection management with custom logic:

   Pinecone::createCollection('analytics-collection', [
       'indexes' => ['user-vectors', 'product-vectors'],
       'metadata' => ['purpose' => 'analytics']
   ]);
   

3. Event Listeners Listen to Pinecone events (e.g., Pinecone\Events\VectorUpserted):

   Pinecone::upsert(...)->then(function ($response) {
       event(new VectorSynced($response));
   });
   

4. Mocking for Tests Use PineconeMock for testing:

   Pinecone::shouldReceive('query')->once()->andReturn([...]);
   

5. Async Operations For long-running operations (e.g., index creation), use the async flag:

   Pinecone::createIndex('large-index', [...], ['async' => true]);
   Pinecone::waitForIndex('large-index'); // Poll until ready