Omines Datatables Elasticsearch Adapter Bundle Laravel Package

Getting Started

Minimal Setup

1. Install the Bundle

   composer require e1on/omines-datatables-elasticsearch-adapter-bundle
   

   Enable the bundle in config/bundles.php:

   return [
       // ...
       E1on\OminesDatatablesElasticsearchAdapterBundle\OminesDatatablesElasticsearchAdapterBundle::class => ['all' => true],
   ];
   

2. Configure Elasticsearch Client Add Elasticsearch client configuration to config/packages/e1on_omines_datatables_elasticsearch_adapter.yaml:

   e1on_omines_datatables_elasticsearch_adapter:
       clients:
           default:
               host: 'elasticsearch'
               port: 9200
   

3. First Use Case: Basic DataTable Define a DataTable with Elasticsearch adapter in a controller or service:

   use E1on\OminesDatatablesElasticsearchAdapterBundle\ElasticaAdapter;
   
   $table = $this->createDataTable()
       ->setName('logs') // Required: Unique name per table
       ->add('timestamp', DateTimeColumn::class, ['field' => '@timestamp'])
       ->add('message', TextColumn::class, ['globalSearchable' => true])
       ->createAdapter(ElasticaAdapter::class, [
           'client' => 'default', // Uses configured client
           'index' => 'logs-*',
       ]);
   

Where to Look First

• Bundle Docs: GitHub README for quick-start examples.
• Omines DataTables Bundle: Official Docs for column types and DataTable configuration.
• Elastica Docs: Elasticsearch PHP Client for query syntax and client setup.

Implementation Patterns

Core Workflow

1. Define DataTable Structure Use Omines DataTables Bundle to define columns and mappings:

   $table = $this->createDataTable()
       ->setName('users')
       ->add('id', NumberColumn::class, ['field' => 'id', 'orderable' => true])
       ->add('name', TextColumn::class, ['globalSearchable' => true, 'field' => 'name.keyword'])
       ->add('email', EmailColumn::class, ['field' => 'email']);
   

2. Configure Elasticsearch Adapter Pass adapter-specific options to createAdapter():

   ->createAdapter(ElasticaAdapter::class, [
       'client' => 'custom_client', // Named client from config
       'index' => 'users_index',
       'type' => '_doc', // Elasticsearch type (default: '_doc')
       'searchAfter' => true, // Enable search_after for pagination (default: true)
       'size' => 10, // Results per page (default: 10)
       'from' => 0, // Starting offset (ignored with search_after)
       'sort' => ['@timestamp' => ['order' => 'desc']], // Required for search_after
   ]);
   

3. Handle Pagination with search_after The adapter automatically uses search_after for efficient pagination. Ensure your DataTable frontend supports passing the last sorted value for subsequent requests.

4. Global Search Integration Leverage Elasticsearch’s query_string or multi_match for global search:

   ->add('global', TextColumn::class, [
       'globalSearchable' => true,
       'field' => ['name.keyword', 'email', 'description'],
   ]);
   

Integration Tips

• Symfony Forms: Use the DataTable bundle’s form integration to render searchable tables in Symfony forms.
• API Endpoints: Return DataTable responses as JSON for API-driven applications:

  return $this->json($table->getData());
  

• Dynamic Indexes: Use wildcards (e.g., logs-*) or aliases for flexible index targeting.
• Caching: Cache Elasticsearch responses if data changes infrequently:

  ->createAdapter(ElasticaAdapter::class, [
      'client' => 'default',
      'cache' => ['enabled' => true, 'ttl' => 300], // Cache for 5 minutes
  ]);
  

Advanced Patterns

• Custom Mappings: Override default field mappings for complex types:

  ->createAdapter(ElasticaAdapter::class, [
      'mappings' => [
          'properties' => [
              'nested_field' => [
                  'type' => 'nested',
                  'properties' => [...],
              ],
          ],
      ],
  ]);
  

• Aggregations: Use Elasticsearch aggregations for grouped data:

  ->createAdapter(ElasticaAdapter::class, [
      'aggregations' => [
          'status_counts' => ['terms' => ['field' => 'status']],
      ],
  ]);
  

• Async Processing: Offload heavy queries to a message queue (e.g., Symfony Messenger) and return cached results immediately.

Gotchas and Tips

Common Pitfalls

1. Missing setName()

   • Error: DataTable name is required.
   • Fix: Always call setName() with a unique identifier (e.g., 'logs', 'users').

2. search_after Requires Sorting

   • Error: Pagination fails or returns empty results.
   • Fix: Define a sort field in the adapter config (e.g., ['@timestamp' => ['order' => 'desc']]). search_after relies on the last sorted value.

3. Field Mismatches

   • Error: Columns return empty or incorrect data.
   • Fix: Ensure Elasticsearch field names match the DataTable column field option (e.g., field => 'name.keyword' for keyword searches).

4. Index/Type Not Found

   • Error: NoSuchIndexException or NoSuchTypeException.
   • Fix: Verify the index and type (default: _doc) exist in Elasticsearch. Use wildcards (e.g., logs-*) cautiously.

5. Performance Issues

   • Cause: Large size values or unoptimized queries.
   • Fix: Limit size (e.g., size: 10) and use search_after for pagination. Avoid from with large offsets.

Debugging Tips

• Enable Elasticsearch Logging Add to config/packages/monolog.yaml:

  handlers:
      elastica:
          type: stream
          path: "%kernel.logs_dir%/elastica.log"
          level: debug
          channels: ["elastica"]
  

  Then configure the Elastica client to log queries:

  $client = Elastica\Client::create([
      'host' => 'elasticsearch',
      'logger' => new \Monolog\Logger('elastica'),
  ]);
  

• Inspect Raw Queries Use the adapter’s debug mode to dump Elasticsearch queries:

  $table->createAdapter(ElasticaAdapter::class, [
      'client' => 'default',
      'debug' => true, // Dumps query to Symfony profiler
  ]);
  

• Validate DataTable Configuration Use the validate() method to check for errors:

  if (!$table->validate()) {
      throw new \RuntimeException('DataTable validation failed: ' . $table->getErrors());
  }
  

Configuration Quirks

1. Client Configuration

   • The bundle expects a named Elastica client. Configure it in config/packages/e1on_omines_datatables_elasticsearch_adapter.yaml:

     e1on_omines_datatables_elasticsearch_adapter:
         clients:
             custom_client:
                 host: 'elasticsearch:9200'
                 port: 9200
                 timeout: 5
     

   • Reference the client by name in the adapter:

     'client' => 'custom_client'
     

2. search_after Behavior

   • The adapter disables traditional from/size pagination when search_after is enabled (default: true).
   • To use from/size, explicitly set searchAfter: false (not recommended for large datasets).

3. Field Data Types

   • Elasticsearch field types (e.g., keyword, text) must align with DataTable column types:
     • Use TextColumn for text fields.
     • Use KeywordColumn for keyword fields.
     • Use DateTimeColumn for date fields.

Extension Points

1. Custom Query Builders Extend the adapter to inject custom Elasticsearch queries:

   use E1on\OminesDatatablesElasticsearchAdapterBundle\ElasticaAdapter;