Elasticsearch Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require pdphilip/elasticsearch
   

   Add the service provider to bootstrap/providers.php (Laravel 11+) or config/app.php (Laravel 10 and below).

2. Configure .env:

   ES_AUTH_TYPE=http
   ES_HOSTS="http://localhost:9200"
   ES_INDEX_PREFIX=my_app_
   

   Update config/database.php with the Elasticsearch connection details.

3. First Use Case: Extend your Eloquent model with PDPhilip\Elasticsearch\Eloquent\Model:

   use PDPhilip\Elasticsearch\Eloquent\Model;
   
   class UserLog extends Model
   {
       protected $connection = 'elasticsearch';
   }
   

   Run a basic search:

   UserLog::searchTerm('Laravel')->get();
   

Where to Look First

• Documentation: Official Docs
• Artisan Commands: elastic:make for scaffolding models, elastic:re-index for migrations.
• Base Model: PDPhilip\Elasticsearch\Eloquent\Model for core functionality.

Implementation Patterns

Usage Patterns

1. Basic CRUD:

   // Create
   UserLog::create(['user_id' => '123', 'ip' => '192.168.1.1']);
   
   // Read
   UserLog::where('status', 1)->get();
   
   // Update
   UserLog::where('status', 1)->update(['status' => 4]);
   
   // Delete
   UserLog::where('status', 4)->delete();
   

2. Search Queries:

   // Full-text search
   UserLog::searchTerm('Laravel')->get();
   
   // Phrase search with prefix
   UserLog::searchPhrasePrefix('loves espressos and t')->highlight()->get();
   
   // Match query
   UserLog::whereMatch('bio', 'PHP')->get();
   
   // Fuzzy search
   UserLog::whereFuzzy('description', 'qick brwn fx')->get();
   

3. Geo Queries:

   // Geo-distance
   UserLog::whereGeoDistance('location', '10km', [40.7185, -74.0025])->get();
   

4. Relationships:

   // Load relationships (supports SQL and ES models)
   UserLog::with('user')->get();
   

5. Aggregations:

   UserLog::aggregate()
       ->terms('status')
       ->get();
   

Workflows

1. Model Scaffolding:

   php artisan elastic:make UserLog
   

   Generates a model with the correct base class and starter mappingDefinition().

2. Re-indexing:

   php artisan elastic:re-index UserLog
   

   Automates index migration with 9 interactive phases (e.g., mapping analysis, data copy, verification).

3. Dynamic Indices: Use dynamicIndex() in mappingDefinition() to enable dynamic indices:

   public static function mappingDefinition(Blueprint $index): void
   {
       $index->dynamicIndex('user_logs_*');
   }
   

4. Chunking:

   UserLog::chunk(100, function ($records) {
       foreach ($records as $record) {
           // Process records in chunks
       }
   });
   

Integration Tips

1. Hybrid Search: Combine SQL and Elasticsearch queries using relationships:

   // SQL model with ES relationship
   class User extends Model {}
   class UserLog extends PDPhilip\Elasticsearch\Eloquent\Model {
       public function user() { return $this->belongsTo(User::class); }
   }
   

2. Custom Mappings: Define mappings in mappingDefinition():

   public static function mappingDefinition(Blueprint $index): void
   {
       $index->text('bio');
       $index->keyword('status');
       $index->geoPoint('location');
   }
   

3. Highlighting:

   UserLog::searchTerm('Laravel')->highlight(['bio'])->get();
   

4. Pagination:

   UserLog::searchTerm('Laravel')->paginate(10);
   

Gotchas and Tips

Pitfalls

1. Index Naming:

   • Prefix collisions: Ensure ES_INDEX_PREFIX is unique to avoid conflicts with other apps.
   • Dynamic indices: Use dynamicIndex() carefully to avoid unintended index proliferation.

2. Mapping Changes:

   • Breaking changes: Altering field types (e.g., text to keyword) requires re-indexing.
   • Use elastic:re-index to automate this process.

3. Relationships:

   • SQL-to-ES relationships: Ensure the SQL model exists and is properly configured.
   • ES-to-ES relationships: Both models must use the same connection.

4. Performance:

   • Large datasets: Use chunk() or cursor() for pagination to avoid memory issues.
   • Default limit: Set ES_OPT_DEFAULT_LIMIT higher than your typical query size (default: 1000).

5. SSL/TLS:

   • Self-signed certs: Disable SSL verification with ES_OPT_VERIFY_SSL=false (not recommended for production).
   • Cloud deployments: Ensure ES_CLOUD_ID and API keys are correctly configured.

Debugging

1. Query DSL:

   • Enable logging in config/database.php:

     'options' => [
         'logging' => true,
     ],
     

   • Logs will appear in storage/logs/elasticsearch.log.

2. Mapping Errors:

   • Use Schema::compileMapping() to preview mappings before applying:

     $mapping = UserLog::compileMapping();
     dd($mapping);
     

3. Connection Issues:

   • Test connectivity with:

     php artisan elastic:ping
     

   • Verify .env settings match your Elasticsearch cluster configuration.

4. Re-indexing Failures:

   • Check the elastic:re-index command logs for phase-specific errors.
   • Use --force to skip confirmations during debugging:

     php artisan elastic:re-index UserLog --force
     

Tips

1. Artisan Commands:

   • elastic:analyze: Analyze text with Elasticsearch analyzers.
   • elastic:ping: Test connection to the Elasticsearch cluster.
   • elastic:show: Display index mappings and settings.

2. Testing:

   • Use PDPhilip\Elasticsearch\Testing\CreatesApplication for Laravel tests:

     use PDPhilip\Elasticsearch\Testing\CreatesApplication;
     
     class TestCase extends TestCase
     {
         use CreatesApplication;
     }
     

3. Configuration:

   • Bypass Validation: Set ES_OPT_BYPASS_MAP_VALIDATION=true to skip mapping validation (useful for dynamic schemas).
   • Sortable IDs: Enable ES_OPT_ID_SORTABLE=true if you need to sort by _id (requires careful index design).

4. Advanced Queries:

   • Nested Queries:

     UserLog::whereNestedObject('tags', ['key' => 'laravel'])->get();
     

   • Cross-field Search:

     UserLog::searchTerm('Laravel')->fields(['bio', 'description'])->get();
     

5. Extensions:

   • Custom Analyzers: Define analyzers in mappingDefinition():

     $index->analyzer('custom_analyzer', function (Blueprint $analyzer) {
         $analyzer->tokenizer('standard');
         $analyzer->filter('lowercase');
     });
     

   • Scripting: Use Elasticsearch painless scripts for dynamic calculations:

     UserLog::scriptField('score', 'doc["views"].value * 1.5');
     

6. Performance Optimization:

   • Index Settings: Configure refresh intervals in mappingDefinition():

     $index->settings(['index.refresh_interval' => '30s']);
     

   • Bulk Operations: Use bulk() for high-volume inserts/updates:

     UserLog::bulk($records);