Laravel Elasticsearch Laravel Package

Getting Started

Minimal Setup

1. Install the package:

   composer require designmynight/laravel-elasticsearch
   

2. Configure Elasticsearch connection in config/database.php:

   'elasticsearch' => [
       'driver'   => 'elasticsearch',
       'host'     => env('ELASTICSEARCH_HOST', 'localhost'),
       'port'     => env('ELASTICSEARCH_PORT', 9200),
       'database' => env('ELASTICSEARCH_INDEX', 'app_index'),
       'suffix'   => env('ELASTICSEARCH_SUFFIX', '_dev'),
   ],
   

3. Update Model.php to override newEloquentBuilder() and newBaseQueryBuilder() (as shown in the README).
4. Create an index mapping (run once):

   php artisan make:mapping users
   php artisan migrate:mappings --index
   

First Use Case: Basic Search

// Search for active users created in the last 30 days
$users = User::newElasticsearchQuery()
    ->where('active', true)
    ->where('created_at', '>', now()->subDays(30))
    ->get();

Implementation Patterns

1. Query Patterns

Eloquent-Like Queries

Leverage familiar Laravel syntax for Elasticsearch:

// Basic filtering
$results = Product::newElasticsearchQuery()
    ->where('price', '<', 100)
    ->where('category', 'electronics')
    ->orderBy('created_at', 'desc')
    ->limit(50)
    ->get();

// Nested queries (bool queries)
$results = Article::newElasticsearchQuery()
    ->where(function ($query) {
        $query->where('title', 'like', '%laravel%')
              ->orWhere('content', 'like', '%elasticsearch%');
    })
    ->where('published', true)
    ->get();

Aggregations

Build nested aggregations with a fluent API:

$query = Order::newElasticsearchQuery()
    ->aggregation('by_status', 'terms', ['field' => 'status'])
    ->aggregation('by_customer', 'terms', ['field' => 'customer_id'])
    ->aggregation('avg_order_value', 'avg', ['field' => 'total']);

$results = $query->get();
$aggregations = $query->getQuery()->getAggregationResults();

// Access aggregations
$statusCounts = $aggregations['by_status']['buckets'];
$avgValue = $aggregations['avg_order_value']['value'];

Geo Queries

// Find restaurants within 5km of a location
$restaurants = Restaurant::newElasticsearchQuery()
    ->whereGeoDistance('location', [40.7128, -74.0060], 5, 'km')
    ->get();

// Filter by bounding box
$restaurants = Restaurant::newElasticsearchQuery()
    ->whereGeoBoundsIn('location', [
        'top_left' => [40.75, -74.05],
        'bottom_right' => [40.65, -73.95]
    ])
    ->get();

2. Data Management

Indexing Models

Use reindex() to sync data between SQL and Elasticsearch:

// Reindex a single model
$user = User::find(1);
$user->reindex();

// Reindex all users (batch processing)
User::chunk(100, function ($users) {
    foreach ($users as $user) {
        $user->reindex();
    }
});

Scroll API for Large Datasets

// Process 100,000+ records efficiently
$scroll = User::newElasticsearchQuery()
    ->limit(100000)
    ->usingScroll()
    ->get();

foreach ($scroll as $user) {
    // Process each user (e.g., update analytics)
    Analytics::logUserView($user);
}

3. Mappings and Migrations

Create/Update Mappings

# Generate a mapping migration
php artisan make:mapping products --template=product_mapping.json

# Apply migrations and index data
php artisan migrate:mappings --index --swap

Custom Mappings

Define mappings in JSON files (e.g., database/migrations/mappings/product_mapping.json):

{
  "mappings": {
    "properties": {
      "name": { "type": "text", "analyzer": "english" },
      "price": { "type": "float" },
      "tags": { "type": "keyword" },
      "location": { "type": "geo_point" }
    }
  }
}

4. Integration with Laravel Features

Events and Observers

// Sync to Elasticsearch when a model is saved
User::observe(UserObserver::class);

class UserObserver {
    public function saved(User $user) {
        $user->reindex();
    }
}

API Resources

// Return Elasticsearch results in an API response
return new UserResource(
    User::newElasticsearchQuery()
        ->where('role', 'admin')
        ->paginate(10)
);

Queued Indexing

// Avoid blocking requests during indexing
dispatch(new IndexUserJob($user->id));

Gotchas and Tips

Common Pitfalls

1. Index Name Mismatch

   • Issue: Queries fail if the Elasticsearch index name doesn’t match the model’s table name.
   • Fix: Ensure the database config key in database.php matches your index name (or override getElasticsearchIndex() in your model):

     public function getElasticsearchIndex()
     {
         return 'custom_' . $this->getTable();
     }
     

2. Mapping Conflicts

   • Issue: Updating mappings after data is indexed can cause errors.
   • Fix: Use --swap with migrate:mappings to safely replace indices:

     php artisan migrate:mappings --index --swap
     

3. Scroll API Memory Leaks

   • Issue: Forgetting to close() a scroll context can exhaust Elasticsearch resources.
   • Fix: Always wrap scroll usage in a try-finally block:

     $scroll = $query->usingScroll()->get();
     try {
         foreach ($scroll as $item) { /* ... */ }
     } finally {
         $scroll->close();
     }
     

4. Geo Distance Units

   • Issue: Forgetting to specify units (km, mi, m) in whereGeoDistance() returns incorrect results.
   • Fix: Always include the unit:

     ->whereGeoDistance('location', [40.7128, -74.0060], 5, 'km')
     

5. Aggregation Key Collisions

   • Issue: Overwriting aggregation results if keys aren’t unique.
   • Fix: Use descriptive keys (e.g., user_status_aggregation instead of status).

6. Bulk Insert Errors

   • Issue: Silent failures during bulk inserts (e.g., reindex() on invalid data).
   • Fix: Wrap in a try-catch:

     try {
         $model->reindex();
     } catch (\DesignMyNight\Elasticsearch\Exceptions\BulkInsertQueryException $e) {
         Log::error('Elasticsearch bulk insert failed: ' . $e->getMessage());
     }
     

Debugging Tips

1. Raw Query Inspection Use toElasticsearch() to see the generated query:

   $query = User::newElasticsearchQuery()->where('active', true);
   dd($query->toElasticsearch());
   

2. Enable Elasticsearch Logging Add to config/elasticsearch.php:

   'log' => [
       'enabled' => true,
       'level' => 'debug',
       'file' => storage_path('logs/elasticsearch.log'),
   ],
   

3. Test with curl Verify Elasticsearch responses directly:

   curl -XGET 'http://localhost:9200/app_index/_search?pretty' -H 'Content-Type: application/json' -d'
   {
     "query": { "match_all": {} }
   }'
   

Performance Optimization

1. Batch Reindexing Use chunking to avoid timeouts:

   User::chunk(500, function ($users) {
       foreach ($users as $user) {
           $user->reindex();
       }
   });
   

2. Limit Aggregation Fields Reduce payload size by specifying only needed fields:

   ->select('id', 'name', 'status')
   ->aggregation('status_counts', 'terms', ['field' => 'status']);
   

3. Use search_after for Pagination Replace from/size with search_after for deep pagination: