Ecs Logging Laravel Package

Getting Started

Minimal Setup

1. Installation Add the package via Composer:

   composer require elastic/ecs-logging
   

   For Laravel, ensure your logging channel (e.g., monolog) is configured to use the ECS formatter.

2. Basic Usage Inject the Elastic\EcsLogger\EcsLogger into your service or controller:

   use Elastic\EcsLogger\EcsLogger;
   
   public function __construct(private EcsLogger $ecsLogger) {}
   

3. First Log Entry Log a structured event with ECS fields:

   $this->ecsLogger->info('User action', [
       'user' => [
           'id' => 123,
           'username' => 'john_doe',
       ],
       'event' => 'login',
       'http' => [
           'request' => [
               'method' => 'POST',
               'url' => '/api/auth',
           ],
       ],
   ]);
   

4. Key Files

   • Review src/EcsLogger.php for core methods.
   • Check src/Formatter/EcsFormatter.php for field mapping rules.
   • Explore src/Processor/ for built-in processors (e.g., HostnameProcessor, ProcessProcessor).

Implementation Patterns

Structured Logging Workflow

1. Contextual Logging Attach metadata (e.g., request ID, user session) via processors:

   $this->ecsLogger->withContext([
       'request_id' => $request->header('X-Request-ID'),
       'trace_id'   => $request->header('X-Trace-ID'),
   ])->info('Processing request');
   

2. Middleware Integration Use middleware to auto-inject ECS fields (e.g., HTTP context):

   public function handle($request, Closure $next) {
       $this->ecsLogger->withContext([
           'http' => [
               'request' => [
                   'method' => $request->method(),
                   'url' => $request->fullUrl(),
               ],
           ],
       ]);
       return $next($request);
   }
   

3. Error Handling Log exceptions with stack traces and ECS fields:

   try {
       // Risky operation
   } catch (\Exception $e) {
       $this->ecsLogger->error('Database failure', [
           'error' => [
               'type' => get_class($e),
               'message' => $e->getMessage(),
               'stack_trace' => $e->getTraceAsString(),
           ],
       ]);
   }
   

4. Custom Processors Extend Elastic\EcsLogger\Processor\ProcessorInterface to add domain-specific fields:

   class UserProcessor implements ProcessorInterface {
       public function process(array $record): array {
           $record['user'] = auth()->user()?->toArray();
           return $record;
       }
   }
   

   Register via service provider:

   $ecsLogger->addProcessor(new UserProcessor());
   

5. Log Levels Leverage ECS-compliant levels (error, warn, info, debug, trace):

   $this->ecsLogger->debug('Debugging user session', ['user' => auth()->user()]);
   

Gotchas and Tips

Common Pitfalls

1. Field Naming Conflicts

   • ECS reserves specific top-level fields (e.g., @timestamp, message, log.level).
   • Fix: Use nested objects (e.g., custom.field) or prefixes (e.g., app.*).

2. Performance Overhead

   • Excessive processors or deep field nesting can slow logging.
   • Tip: Disable debug-level logging in production and lazy-load heavy processors.

3. Timestamp Handling

   • The formatter defaults to now() for @timestamp. Override if using custom time sources:

   $ecsLogger->setTimestamp(function () {
       return Carbon::now()->toDateTimeString();
   });
   

4. Processor Order Matters Processors execute in registration order. Critical fields (e.g., service.name) should run early.

   • Debug: Temporarily add a dump() in a processor to inspect the record state.

5. Monolog Integration

   • If using Monolog, ensure the ECS formatter is the last processor in the chain to avoid field corruption:

   $handler->pushProcessor(new EcsFormatter());
   

Debugging Tips

1. Inspect Raw Records Temporarily log the raw ECS array to verify structure:

   $this->ecsLogger->info('Debug ECS', ['_raw' => $this->ecsLogger->getProcessor()->process([])]);
   

2. Validate Against ECS Spec Use ECS Validator to check compliance:

   curl -XPOST "localhost:9200/_validate/query?pretty" -H 'Content-Type: application/json' -d'
   {
     "query": {
       "bool": {
         "must": [
           { "exists": { "field": "event.id" } }
         ]
       }
     }
   }'
   

3. Log Level Filtering

   • Use Laravel’s log channels to filter levels (e.g., single channel with level: warning):

   'logging' => [
       'channels' => [
           'ecs' => [
               'driver' => 'single',
               'level' => env('LOG_LEVEL', 'warning'),
               'handler' => Elastic\EcsLogger\Handler\StreamHandler::class,
           ],
       ],
   ],
   

Extension Points

1. Custom Field Mappers Override Elastic\EcsLogger\Formatter\EcsFormatter to map Laravel-specific data (e.g., Illuminate\Database\Eloquent\Model to ECS):

   protected function mapModelToEcs($model): array {
       return [
           'object' => [
               'id' => $model->getKey(),
               'type' => get_class($model),
               'attributes' => $model->toArray(),
           ],
       ];
   }
   

2. Dynamic Field Injection Use Laravel’s app() helper to inject dynamic values (e.g., config, environment):

   $this->ecsLogger->addProcessor(function (array $record): array {
       $record['service'] = [
           'version' => config('app.version'),
           'environment' => app()->environment(),
       ];
       return $record;
   });
   

3. Log Sampling Implement probabilistic sampling for high-volume logs:

   $this->ecsLogger->addProcessor(function (array $record) {
       if (rand(0, 99) > 50) { // 50% sampling
           return $record;
       }
       return ['@metadata' => ['sampled' => false]];
   });
   

4. Log Enrichment via Observers Attach ECS fields to Eloquent models via observers:

   class UserObserver {
       public function saved(User $user) {
           \Log::ecs('User updated', [
               'user' => [
                   'id' => $user->id,
                   'action' => 'update',
               ],
           ]);
       }
   }