Elasticsearch Eloquent Laravel Package

Getting Started

Minimal Setup

1. Installation:

   composer require isswp101/elasticsearch-eloquent
   

   Ensure your composer.json meets the requirements (PHP 8.0+, Elasticsearch 7.0+).

2. Define a Model: Extend BaseElasticsearchModel and set $index (required) and optionally $type (for Elasticsearch 7.x compatibility).

   use Isswp101\Persimmon\Models\BaseElasticsearchModel;
   
   class Product extends BaseElasticsearchModel {
       protected string $index = 'products';
       protected ?string $type = 'product';
   }
   

3. First Query: Use Eloquent-like syntax to interact with Elasticsearch:

   // Create
   $product = Product::create(['name' => 'Laptop', 'price' => 999]);
   
   // Query
   $results = Product::where('price', '>', 500)->get();
   

4. Configuration: Publish the config file (if needed) and customize Elasticsearch client settings:

   php artisan vendor:publish --provider="Isswp101\Persimmon\Providers\PersimmonServiceProvider"
   

Implementation Patterns

Core Workflows

1. CRUD Operations:

   • Create: Use create() or save() (mirrors Eloquent).

     $product = Product::create(['name' => 'Phone', 'stock' => 10]);
     

   • Update: Assign attributes and call save().

     $product->price = 799;
     $product->save();
     

   • Delete: Use delete() or destroy().

     Product::find(1)->delete();
     

2. Querying:

   • Basic Queries: Chain methods like Eloquent.

     Product::where('name', 'like', '%Laptop%')->orderBy('price', 'desc')->get();
     

   • Advanced Queries: Use Elasticsearch-specific query builders.

     $query = Product::query()
         ->addShould(['name' => 'Phone'])
         ->addMust(['price' => ['gte' => 500]])
         ->get();
     

   • Aggregations:

     $results = Product::query()
         ->addAggregation('avg_price', 'avg', 'price')
         ->get();
     

3. Relationships:

   • Define relationships using belongsTo, hasMany, etc., but ensure the related models are also Elasticsearch models.

     class Order extends BaseElasticsearchModel {
         public function products() {
             return $this->hasMany(Product::class);
         }
     }
     

   • Note: Relationships are fetched via nested queries or joins (if configured).

4. Bulk Operations:

   • Use insert() for bulk creates or update() for bulk updates.

     Product::insert([
         ['name' => 'Tablet', 'price' => 300],
         ['name' => 'Monitor', 'price' => 200],
     ]);
     

5. Index Management:

   • Create/update indices via artisan commands:

     php artisan elasticsearch:create-index products
     php artisan elasticsearch:update-mapping products
     

Integration Tips

1. Hybrid Models (SQL + Elasticsearch):

   • Extend Laravel’s Model and BaseElasticsearchModel to sync data between databases.
   • Use observers or model events (saved, deleted) to keep Elasticsearch in sync.

     class Product extends Model {
         use \Isswp101\Persimmon\Traits\ElasticsearchTrait;
     
         protected $elasticsearchEnabled = true;
     }
     

2. Custom Query Builders:

   • Extend the query builder to add domain-specific methods:

     class ProductQueryBuilder extends \Isswp101\Persimmon\Query\Builder {
         public function inStock() {
             return $this->where('stock', '>', 0);
         }
     }
     

   • Bind the builder to your model in boot():

     Product::addGlobalScope(new CustomQueryScope());
     

3. Caching:

   • Cache frequent queries or results to reduce Elasticsearch load.

     $products = Cache::remember('featured_products', now()->addHours(1), function() {
         return Product::where('featured', true)->get();
     });
     

4. Testing:

   • Use ElasticsearchFake for unit tests (if available) or reset indices between tests.

     public function tearDown(): void {
         Product::resetIndex();
         parent::tearDown();
     }
     

Gotchas and Tips

Pitfalls

1. Index/Type Mismatch:

   • Elasticsearch 7.x+ uses index-only naming (no types). If $type is set, the package may fail silently or throw errors.
   • Fix: Omit $type or ensure your Elasticsearch version is compatible.

2. Data Type Conflicts:

   • Elasticsearch requires explicit mapping for fields (e.g., integer, keyword, text).
   • Fix: Define mappings in config/elasticsearch.php or use update-mapping artisan command.

     'mappings' => [
         'properties' => [
             'price' => ['type' => 'float'],
             'name' => ['type' => 'text', 'fields' => ['raw' => ['type' => 'keyword']]],
         ],
     ],
     

3. Case Sensitivity:

   • where() clauses are case-sensitive by default for keyword fields.
   • Fix: Use lower() or upper() in queries or normalize data before indexing.

4. Relationship Loading:

   • Nested relationships may not load automatically. Use with() explicitly:

     $order = Order::with('products')->find(1);
     

   • Note: Deeply nested relationships can bloat queries; consider lazy-loading.

5. Bulk Operation Limits:

   • Elasticsearch has limits on bulk request sizes (~10MB or 1,000 operations).
   • Fix: Chunk large operations:

     $products->chunk(500, function ($chunk) {
         Product::insert($chunk->toArray());
     });
     

6. Connection Issues:

   • The package assumes a default Elasticsearch client. If using custom clients, ensure they’re properly configured in createPersistence().
   • Fix: Verify client settings and handle exceptions:

     try {
         $product->save();
     } catch (\Elasticsearch\Common\Exceptions\ElasticsearchException $e) {
         Log::error('Elasticsearch error: ' . $e->getMessage());
     }
     

Debugging Tips

1. Query Logging:

   • Enable query logging in config/elasticsearch.php:

     'logging' => true,
     

   • Check logs for raw Elasticsearch queries (useful for troubleshooting).

2. Kibana Validation:

   • Use Kibana’s Dev Tools to test queries manually before implementing them in Laravel.

3. Model Events:

   • Override boot() to log or validate data before Elasticsearch operations:

     protected static function boot() {
         parent::boot();
         static::saving(function ($model) {
             if (empty($model->name)) {
                 throw new \Exception('Product name cannot be empty.');
             }
         });
     }
     

4. Index Existence:

   • Always check if an index exists before operations:

     if (!Product::indexExists()) {
         Product::createIndex();
     }
     

Extension Points

1. Custom Persistence:

   • Override createPersistence() to use a custom Elasticsearch client (e.g., with retry logic or middleware).

     public function createPersistence(): PersistenceContract {
         $client = Elasticsearch\ClientBuilder::create()
             ->setHosts(['http://es:9200'])
             ->setRetryOnConflict(3)
             ->build();
         return new Persistence($client);
     }
     

2. Query Scopes:

   • Add reusable scopes to your model:

     class Product extends BaseElasticsearchModel {
         public function scopeOnSale($query) {
             return $query->where('discount', '>', 0);
         }
     }
     

     Usage:

     Product::onSale()->get();
     

3. Dynamic Indexing:

   • Use a dynamic index name based on environment or tenant:

     protected string $index = config('app.env') === 'production' ? 'products_prod' : 'products_dev';
     

4. Webhook Triggers:

   • Integrate with Laravel events to trigger Elasticsearch updates on SQL changes:

     Product::observe(ProductObserver::class);
     
     class ProductObserver {
         public function saved(Product $product) {
             if ($