Monolog Ecs Formatter Laravel Package

Getting Started

The package fills a narrow but real gap: Monolog’s stock JSON formatter dumps everything into context and extra, which forces Elasticsearch to guess at mappings and makes dashboards brittle. This formatter promotes ECS-style fields (error.type, service.name, user.id, trace.id, labels.*, metrics.*, and as of v0.2.1 network.direction) to top-level keys so you can define explicit mappings and query them directly.

Install and confirm what you actually got. The package is a Monolog formatter plus a Symfony bundle. Laravel will not auto-register the bundle — you only care about the formatter class. Before wiring it in, read the repository source (src/) and the README to confirm the exact formatter class name, constructor options, and supported Monolog version. This package has single-digit stars and no dependents, so pin an exact version and treat the source as the spec.

composer require herdwatch-oss/monolog-ecs-formatter:0.2.1

Where to look first, in order: the README’s config snippet → the formatter class (__construct signature tells you every option) → the value-object classes (metrics, labels, text, tags, service, user, tracing, error, and now network) since those define the field names your log calls must produce → the test suite, which is the most reliable documentation of expected input shapes. For v0.2.1 specifically, the two files worth reading are the Network value object and the NetworkDirection enum: the enum is a closed set pinned to ECS 8.11, which tells you exactly which strings are legal and whether the value object expects an enum instance or a raw string.

First use case: get one channel emitting ECS JSON to a file. Don’t start with Elasticsearch. Add a dedicated channel in config/logging.php, write to storage/logs/ecs.log, and eyeball the output:

'ecs' => [
    'driver' => 'monolog',
    'level' => env('LOG_LEVEL', 'debug'),
    'handler' => Monolog\Handler\StreamHandler::class,
    'handler_with' => ['stream' => storage_path('logs/ecs.log')],
    'formatter' => \Herdwatch\MonologEcsFormatter\Formatter\EcsFormatter::class,
    'formatter_with' => [
        // copy mode: keeps channel, level_name, level, datetime alongside ECS fields
        'copyLegacyKeys' => true,
    ],
],

Change the formatter options to match the real constructor (named args may differ). Once tail -f storage/logs/ecs.log shows @timestamp, log.level, message, and your promoted fields at the top level, you can point the same channel at a shipper or an Elasticsearch handler and the mapping work is already done.

Implementation Patterns

Laravel channel wiring (the main integration point). Laravel’s monolog driver accepts formatter, formatter_with, handler, handler_with, and processors. If formatter_with doesn’t cover your needs (e.g. you need to inject a service-name string from config('app.name')), use the closure form:

'formatter' => function () {
    return new \Herdwatch\MonologEcsFormatter\Formatter\EcsFormatter(
        copyLegacyKeys: app()->environment() !== 'production',
    );
},

Stack channels for migration. The killer feature is the default copy mode: legacy keys (channel, level_name, level, datetime) are retained alongside the ECS fields. That means you can flip a service to the new formatter without breaking any existing Kibana saved search, alert, or Grafana panel. Pattern:

1. Ship with copy mode on in staging and production.
2. Verify new ECS-field queries return the same results as old level_name queries.
3. Once every dashboard is migrated, turn copy mode off to stop paying for duplicate keys in your index.

Keep the same channel name during this transition so you don’t accidentally split indices.

Structure your log calls to match the value objects. The formatter can only promote fields that exist. Establish a house convention early — a small helper or trait that builds the context array once:

Log::info('payment captured', [
    'service' => ['name' => 'billing', 'environment' => app()->environment(), 'version' => config('app.version')],
    'labels' => ['gateway' => 'stripe', 'tenant' => tenant()->id],
    'metrics' => ['amount' => $order->total, 'duration_ms' => $elapsed],
    'user' => ['id' => (string) $request->user()?->id],
    'trace' => ['id' => $request->header('X-Request-Id')],
    'network' => ['direction' => NetworkDirection::Outbound], // v0.2.1+
    'error' => $e ? ['type' => get_class($e), 'message' => $e->getMessage()] : null,
]);

Exact nesting depends on what the value objects accept — validate against the source rather than guessing. In particular, check whether network.direction takes a NetworkDirection case or a plain string before you sprinkle it through your codebase.

Use network.direction where it actually means something. The new field is genuinely useful in two places in a typical Laravel app, and noise everywhere else:

• Outbound HTTP telemetry. A Guzzle/Laravel Http:: middleware or a Log::withContext wrapper around Http::client() calls, logging network.direction = outbound plus target host and status. This is where the field earns its keep — you can chart outbound error rates per dependency with a single filter.
• Inbound request logging. A terminating middleware that logs the request summary with inbound. Pairs well with trace.id from the same middleware.

Don’t add it to domain/business log lines. There is no direction on “payment captured”; populating unknown everywhere just bloats documents and makes the real signal harder to find.

Inject ambient context with a processor, not with every call site. service.*, trace.*, user.*, and the inbound network.direction are identical for every record in a request. A Monolog processor that adds service / trace / network arrays to the record is far less error-prone than trusting 200 developers to pass it. Register it with Laravel’s tap option on the channel ('tap' => [App\Logging\EcsContextTap::class]) so the processor is only attached to this channel.

Correlation and errors. Push trace.id from a middleware (request ID, W3C traceparent, or OpenTelemetry span id). For exceptions, prefer passing the exception through Monolog’s native path and let the formatter map it to error.* — but check how it handles stack traces, since error.stack_trace can be multi-kilobyte and will blow up your index if unbounded. If the formatter doesn’t truncate, add a processor that trims.

Testing. Log::fake() intercepts before the handler, so it will not exercise the formatter. To test output, either instantiate the formatter directly and assert on format() against a hand-built LogRecord, or swap the channel to a TestHandler and assert on the formatted array. A small unit test that pins your field names — including that NetworkDirection::Inbound serialises to the string inbound and not to "Inbound" or an enum object — is worth it here; it’s your only protection against an upstream release silently renaming a key.

Gotchas and Tips

The Symfony bundle is noise in Laravel. Don’t try to register it. Check composer.json for symfony/* dependencies it drags in; they’re usually harmless but confirm nothing conflicts with your Laravel/Symfony version constraints. Also verify the Monolog constraint — Monolog 3 changed the LogRecord API, and a formatter written for Monolog 2 will fatally error on format(array $record).

The ECS version pin is now explicit, and that cuts both ways. NetworkDirection is pinned to ECS 8.11. That’s good — you know the vocabulary is stable and spec-compliant. The cost is that moving to a newer ECS (or using a field introduced after 8.11) requires a package bump, and the package is young. If you need a field the value objects don’t model, use the processor escape hatch rather than forking the enum. Also double-check the enum’s values against your own ECS expectations: internal/external are in the set but many Kibana dashboards and ECS consumers only ever filter on inbound/outbound, so don’t build alerting on the rarer cases without verifying they’re indexed and queryable.

Enum vs string at the call site. If Network type-hints NetworkDirection, passing 'outbound' throws a TypeError — good, fail fast. If it accepts string and validates internally, watch for silently coerced/ignored values. Either way, prefer the enum in your own code: it gives IDE completion and makes an ECS vocabulary change a compile-time-ish error rather than a runtime data-quality bug. And confirm enums are actually serialised to their ->value — a json_encode of a raw enum object yields either the value or {} depending on whether it implements JsonSerializable, and the failure mode is a silently empty network.direction.

Mapping explosions are the failure this package prevents — don’t reintroduce them. labels.* and metrics.* are attractive but if you put unbounded keys there (user IDs, order IDs, URLs, raw exception messages), Elasticsearch creates a field per key. Keep labels to a fixed, low-cardinality set (gateway, tenant tier, queue name) and use flattened or dynamic: false in your index template for labels and metrics as a safety net. Anything high-cardinality belongs in message or a keyword field you’ve deliberately mapped. Same applies to any hostname/URL you log next to network.direction — that’s exactly the kind of field that quietly explodes a mapping.

Copy mode doubles your key space. It’s a migration tool, not a permanent setting. Two field names for the same value inevitably leads to dashboards that disagree; set a date to remove it.

Timezone and timestamp drift. Confirm whether the formatter emits @timestamp in UTC with ISO-8601, and whether it also emits datetime from Monolog (which may be in app.timezone). Mixing the two across services produces logs that sort wrong in Kibana. Force UTC at the app level and map @timestamp explicitly as a date field with the right format.

level_name vs log.level casing. ECS expects lowercase-ish syslog severities in log.level while Monolog’s level_name is INFO, WARNING. Don’t assume your existing level_name: ERROR filters translate one-to-one; write the replacement query, run both, compare counts before deleting anything.

Debugging workflow. When output looks wrong, forget the whole pipeline: build a LogRecord in php artisan tinker, call (new EcsFormatter(...))->format($record), and json_encode the result. Nine times out of ten the problem is the input context shape (nested vs flat, null values, Log::info($msg, $context) with the context as the second argument, not string-interpolated). For v0.2.1, run that same loop once with a Network object in the context to confirm the field lands at network.direction — not nested under context.network, and not stringified as an object dump.

Nulls and empty arrays. Decide whether you want ECS keys present-but-null or omitted. Present-but-null keeps mappings stable but inflates storage; omitted keys mean a field can vanish from an index until the first record that has it, which surprises people writing aggregations. This bites network.direction hardest, because only a fraction of your log lines will carry it — make sure your index template maps it up front instead of waiting for the first inbound request to create the field dynamically.

Truncation and size limits. Exception traces, request bodies, and query strings are the usual suspects for pushing documents past your bulk-index limits. Check what the formatter does with error payloads and add truncation upstream if it just passes them through.

Pin and vendor. Single-digit stars, zero dependents, a small set of contributors, and a 0.x version number — the public API is not promised stable. Use an exact version constraint ("0.2.1", not "^0.2"), keep the channel config in one place, and consider forking into your own namespace if the field names matter to more than a handful of dashboards — you want to control the schema, not inherit it. Also sanity-check the license (MIT is fine) and that the package doesn’t ship telemetry, network calls, or a service provider with side effects.

Extension points. If the value objects don’t cover a field you need, a Monolog processor that adds the key to $record['context'] is the lowest-friction escape hatch, assuming the formatter’s promotion logic passes unknown keys through. Verify that it does — some formatters drop anything they don’t recognise, which would silently discard your custom fields. This matters more now that the package has begun curating a closed vocabulary: each release adds a handful of modelled fields, and the temptation to fork-the-enum rather than use a processor grows with it.