Explorer Laravel Package

Technical Evaluation

Architecture Fit

• Strengths:

  • Seamless Laravel Scout Integration: Leverages Laravel Scout’s existing search abstractions (search(), paginate(), cursor()), reducing boilerplate and maintaining consistency with Laravel’s ecosystem.
  • Elasticsearch-First Features: Exposes advanced Elasticsearch capabilities (e.g., aggregations, sorting, pagination, text analysis) while abstracting low-level complexity. Ideal for applications requiring full-text search, analytics, or complex queries.
  • Modular Design: Separates concerns (application logic, domain models, infrastructure) to facilitate testing, maintenance, and future extensibility.
  • Index Aliases Support: Enables zero-downtime index updates and rollbacks, critical for production-grade search systems.

• Fit for Use Cases:

  • High-Performance Search: Replace database LIKE queries with Elasticsearch’s inverted index for sub-millisecond response times.
  • Analytics/Insights: Use aggregations (e.g., terms, date_histogram) for dashboards or reporting.
  • Multi-Tenant Search: Isolate indices per tenant or environment via configuration.
  • Hybrid Search: Combine keyword, vector (BM25), and fuzzy matching for nuanced queries.

• Limitations:

  • Not a Replacement for Database: Requires denormalized data in Elasticsearch (e.g., toSearchableArray()). Eventual consistency must be managed (e.g., via Laravel Scout’s searchable() events).
  • Elasticsearch Dependency: Adds operational overhead (cluster management, scaling, backups). Not suitable for simple CRUD apps.
  • Learning Curve: Elasticsearch’s query DSL and mapping system require familiarity for advanced use cases.

Integration Feasibility

• Prerequisites:

  • Laravel Scout: Must be installed and configured (config/scout.php driver set to elastic).
  • Elasticsearch Cluster: Local (Docker), cloud (Elastic Cloud), or self-hosted. Version compatibility: Tested with Elasticsearch 7.x/8.x (see connection docs).
  • PHP Extensions: php-curl, php-json (required by Elasticsearch PHP client).

• Migration Path:

  1. Pilot Phase:
     • Start with a single model (e.g., Post) to validate performance and query accuracy.
     • Use scout:import to backfill existing data.
  2. Incremental Rollout:
     • Gradually migrate models to Elasticsearch, leveraging Scout’s searchable() events to sync changes.
     • Use index aliases to avoid downtime during updates.
  3. Deprecation:
     • Replace database queries with Scout methods (e.g., Post::search('term')->get()).
     • Phase out raw Elasticsearch queries in favor of Explorer’s abstractions.

• Compatibility:

  • Laravel Versions: Tested with Laravel 8+ (Scout v8+). Backward compatibility may require adjustments for older versions.
  • Elasticsearch Versions: Supports 7.x/8.x. TLS/SSL configurations are version-dependent (see connection.md).
  • Database Agnostic: Works with any Laravel-supported database (MySQL, PostgreSQL, etc.), but requires manual syncing of changes.

Technical Risk

• Data Consistency:

  • Risk: Elasticsearch and database may diverge if searchable() events fail or data isn’t reindexed.
  • Mitigation:
    • Use Laravel’s queue system to process searchable events asynchronously.
    • Implement health checks (e.g., scout:flush) to verify index counts match database records.
    • Consider Explorer’s toSearchableArray() hooks for custom sync logic.

• Performance:

  • Risk: Poorly configured mappings or queries can degrade performance (e.g., match_all queries, unoptimized analyzers).
  • Mitigation:
    • Profile queries using ElasticEngine::debug() and Kibana’s Dev Tools.
    • Limit index size via pagination (->paginate(10)) or cursor-based scrolling.
    • Use keyword mappings for exact matches (e.g., IDs, categories) and text for full-text.

• Operational Complexity:

  • Risk: Elasticsearch clusters require tuning (shards, replicas, JVM heap size).
  • Mitigation:
    • Start with a single-node Docker setup for development (see quickstart.md).
    • Monitor cluster health via Kibana or Elasticsearch APIs.
    • Use Explorer’s logger config to debug connection issues.

• Vendor Lock-in:

  • Risk: Custom mappings or queries may become hard to migrate away from Elasticsearch.
  • Mitigation:
    • Design mappings to be portable (e.g., avoid Elasticsearch-specific features like nested types unless necessary).
    • Abstract query logic behind service layers to ease future swaps (e.g., Algolia, Meilisearch).

Key Questions

1. Search Requirements:
   • What percentage of queries are full-text vs. exact-match (e.g., IDs, tags)?
   • Are aggregations (e.g., faceted search) or geospatial queries needed?
2. Data Volume:
   • How many records per model? Elasticsearch sharding may be needed for >10M documents.
   • What’s the write throughput (e.g., real-time updates vs. batch imports)?
3. Operational Constraints:
   • Is the team comfortable managing Elasticsearch clusters, or is a managed service (e.g., Elastic Cloud) preferred?
   • What’s the acceptable latency for index updates (e.g., near-real-time vs. hourly syncs)?
4. Cost:
   • For cloud deployments, estimate costs based on node types, storage, and query volume.
5. Fallback Strategy:
   • How should the system behave if Elasticsearch is unavailable (e.g., fallback to database LIKE queries)?

Integration Approach

Stack Fit

• Laravel Ecosystem:

  • Scout Integration: Explorer replaces Scout’s default drivers (e.g., database, algolia) with Elasticsearch. Leverage existing Scout features like:
    • search(), paginate(), cursor() for consistent API.
    • toSearchableArray() for custom data transformation.
    • searchable() events for syncing changes.
  • Service Providers: Explorer registers its own service provider (JeroenG\Explorer\ExplorerServiceProvider), which integrates with Laravel’s IoC container.
  • Artisan Commands: Provides CLI tools for index management (scout:import, elastic:update, scout:delete-index).

• Elasticsearch Stack:

  • Client: Uses the official elastic/elasticsearch-php client, which supports all Elasticsearch APIs.
  • Query DSL: Exposes Elasticsearch’s full query capabilities (e.g., bool, match, aggregations) via Scout methods or raw queries.
  • Infrastructure: Works with any Elasticsearch deployment (local, cloud, or hybrid).

• Testing:

  • Unit Testing: Mock Elasticsearch responses using FakeResponse (see testing.md) to avoid dependency on a live cluster.
  • Feature Testing: Test search flows end-to-end with Laravel’s HTTP tests.

Migration Path

1. Preparation:

   • Setup Elasticsearch: Deploy a cluster (Docker, cloud, or self-hosted) and configure Kibana for monitoring.
   • Configure Explorer:
     • Publish config: php artisan vendor:publish --tag=explorer.config.
     • Set driver => 'elastic' in config/scout.php.
     • Define mappings in config/explorer.php (e.g., posts_index).
   • Install Dependencies:

     composer require jeroen-g/explorer
     composer require elastic/elasticsearch
     

2. Pilot Phase:

   • Select a Model: Start with a low-traffic model (e.g., Post, Product).
   • Define Mappings: Map fields to Elasticsearch types (e.g., title => 'text', published_at => 'date').
   • Backfill Data:

     php artisan scout:import "App\Models\Post"
     

   • Test Queries:

     $results = Post::search('laravel')->paginate(10);
     

   • Validate: Compare results with existing database queries for accuracy.

3. Incremental Rollout:

   • Add Models: Gradually migrate other models, reusing mappings where possible.
   • Sync Strategy:
     • For real-time updates, use searchable() events.
     • For batch updates, schedule scout:import via Laravel’s scheduler.
   • Index Aliases: Use elastic:update to manage aliases for zero-downtime updates.

4. Deprecation:

   • Replace direct database queries with Scout methods.
   • Deprecate legacy search logic in favor of Explorer’s abstractions.
   • Monitor performance and adjust mappings/queries as needed.

**