Lucene Search Bundle Laravel Package

Getting Started

Minimal Setup

1. Installation

   composer require egeloen/lucene-search-bundle
   

   Add to config/bundles.php:

   return [
       // ...
       Egeloen\LuceneSearchBundle\EgeloenLuceneSearchBundle::class => ['all' => true],
   ];
   

2. Configuration Define indexes in config/packages/egeloen_lucene_search.yaml:

   egeloen_lucene_search:
       indexes:
           default:
               path: '%kernel.project_dir%/var/lucene/default'
               fields:
                   - { name: 'title', type: 'text' }
                   - { name: 'content', type: 'text' }
                   - { name: 'created_at', type: 'date' }
   

3. First Use Case Index a model (e.g., Article):

   use Egeloen\LuceneSearchBundle\Indexer\IndexerInterface;
   
   class ArticleService {
       public function __construct(private IndexerInterface $indexer) {}
   
       public function indexArticle(Article $article) {
           $this->indexer->index('default', $article->toArray());
       }
   }
   

   Query the index:

   use Egeloen\LuceneSearchBundle\Searcher\SearcherInterface;
   
   class ArticleSearch {
       public function __construct(private SearcherInterface $searcher) {}
   
       public function search(string $query) {
           return $this->searcher->search('default', $query);
       }
   }
   

Implementation Patterns

Workflows

1. Indexing Strategy

   • Batch Indexing: Use IndexerInterface::index() for single documents or indexMany() for bulk operations.
   • Event-Driven Indexing: Trigger indexing via Doctrine lifecycle events (e.g., postPersist, postUpdate):

     // src/EventListener/ArticleIndexListener.php
     use Egeloen\LuceneSearchBundle\Indexer\IndexerInterface;
     
     class ArticleIndexListener {
         public function __construct(private IndexerInterface $indexer) {}
     
         public function postPersist(Article $article) {
             $this->indexer->index('default', $article->toArray());
         }
     }
     

2. Querying Patterns

   • Basic Search:

     $results = $this->searcher->search('default', 'lucene symfony');
     

   • Advanced Queries (using Lucene syntax):

     $query = 'title:"Symfony" AND (content:lucene OR content:search)';
     $results = $this->searcher->search('default', $query);
     

   • Pagination:

     $results = $this->searcher->search('default', $query, 0, 10); // offset, limit
     

3. Integration with Laravel Controllers

   • Inject services into controllers:

     use Egeloen\LuceneSearchBundle\Searcher\SearcherInterface;
     
     class ArticleController extends Controller {
         public function __construct(private SearcherInterface $searcher) {}
     
         public function search(Request $request) {
             $query = $request->input('q');
             $results = $this->searcher->search('default', $query);
             return view('articles.search', compact('results'));
         }
     }
     

4. Hybrid Search (Database + Lucene)

   • Use Lucene for full-text search and the database for exact matches or metadata:

     // Pseudocode for hybrid search
     $luceneResults = $this->searcher->search('default', $query);
     $dbResults = Article::where('title', 'like', "%{$query}%")->get();
     $mergedResults = $luceneResults->merge($dbResults);
     

Gotchas and Tips

Pitfalls

1. Index Path Permissions

   • Ensure the Lucene index directory (var/lucene/) is writable by the web server user:

     chmod -R 775 var/lucene/
     

   • Debugging: Check Symfony logs (storage/logs/dev.log) for Permission denied errors.

2. Field Mapping Mismatches

   • If queries return no results, verify field names/types in config/packages/egeloen_lucene_search.yaml match your data.
   • Example: A date field in config won’t work with a string value in your data.

3. Lucene Index Corruption

   • If the index becomes corrupted, delete the directory and reindex:

     rm -rf var/lucene/default/
     

   • Reindex all data via a console command or batch process.

4. Case Sensitivity

   • Lucene is case-sensitive by default. Use TextField with lowercase analyzer for case-insensitive searches:

     fields:
         - { name: 'title', type: 'text', analyzer: 'lowercase' }
     

5. Memory Usage

   • Large indexes may cause high memory usage. Optimize by:
     • Using smaller batch sizes for indexMany().
     • Implementing a queue system (e.g., Laravel Queues) for background indexing.

Debugging Tips

1. Query Debugging

   • Use the raw Lucene query syntax to debug issues:

     $query = 'title:"Symfony"^2 content:lucene';
     // ^2 boosts the title field
     

   • Test queries directly in a Lucene tool like Luke or Solr Admin UI if using Solr.

2. Logging

   • Enable debug mode in config/packages/dev/egeloen_lucene_search.yaml:

     egeloen_lucene_search:
         debug: true
     

   • Check logs for indexer/searcher operations.

3. Configuration Overrides

   • Override bundle config per environment (e.g., config/packages/prod/egeloen_lucene_search.yaml):

     egeloen_lucene_search:
         indexes:
             default:
                 path: '%kernel.project_dir%/var/lucene/prod_default'
     

Extension Points

1. Custom Analyzers

   • Extend the bundle to support custom analyzers (e.g., stemming, synonyms):

     // src/Analyzer/CustomAnalyzer.php
     use Egeloen\LuceneSearchBundle\Analyzer\AnalyzerInterface;
     
     class CustomAnalyzer implements AnalyzerInterface {
         public function analyze(string $text): array {
             // Custom logic (e.g., stemming)
             return [$text]; // Simplified
         }
     }
     

   • Register in config:

     egeloen_lucene_search:
         analyzers:
             custom:
                 class: App\Analyzer\CustomAnalyzer
     

2. Event Listeners

   • Hook into indexing/searching events (e.g., pre/post index, search):

     // src/EventListener/LuceneEventListener.php
     use Egeloen\LuceneSearchBundle\Event\IndexEvent;
     
     class LuceneEventListener {
         public function onIndex(IndexEvent $event) {
             // Modify document before indexing
             $event->setDocument($event->getDocument()->merge(['custom_field' => 'value']));
         }
     }
     

   • Bind in services.yaml:

     services:
         App\EventListener\LuceneEventListener:
             tags:
                 - { name: kernel.event_listener, event: egeloen_lucene.index, method: onIndex }
     

3. Custom Search Results

   • Transform raw Lucene results into Eloquent models or DTOs:

     class ArticleSearchResult {
         public static function fromLuceneResult(array $result): self {
             return new self(
                 id: $result['id'],
                 title: $result['title'],
                 score: $result['score']
             );
         }
     }