Laravel Monolog Kinesis Laravel Package

Getting Started

composer require pod-point/laravel-monolog-kinesis
php artisan vendor:publish --tag=monolog-kinesis-config   # or --provider="PodPoint\MonologKinesis\MonologKinesisServiceProvider"

The package registers a kinesis Monolog driver with Laravel's Log manager. Everything you touch day-to-day lives in two places: config/logging.php (channel definition) and config/monolog-kinesis.php (stream/region/credentials defaults).

Add a channel:

'kinesis' => [
    'driver'  => 'kinesis',
    'stream'  => env('KINESIS_LOG_STREAM', 'laravel-logs'),
    'region'  => env('AWS_DEFAULT_REGION', 'eu-west-1'),
    'level'   => env('LOG_LEVEL', 'debug'),
    // omit or set to null to use the AWS SDK default credential chain (v4.0.2+)
    'credentials' => [
        'key'    => env('AWS_ACCESS_KEY_ID'),
        'secret' => env('AWS_SECRET_ACCESS_KEY'),
    ],
],

First use case: pipe application logs into a Kinesis stream fronted by Firehose → S3 → Athena, so you get cheap, queryable, structured logs without running an ELK stack. Start by pointing a stack channel at it so you don't lose local visibility:

'stack' => [
    'driver'   => 'stack',
    'channels' => ['single', 'kinesis'],
],

Then LOG_CHANNEL=stack in staging, and a smoke test: php artisan tinker → Log::channel('kinesis')->info('hello', ['foo' => 'bar']), then read the record back out of the stream.

Implementation Patterns

Stack + environment gating. Never make kinesis the sole channel. Keep single/daily in the stack locally and in CI (no AWS creds, no 403 noise from the handler), and only switch LOG_CHANNEL in deployed environments.

One stream per concern, driven by env. KINESIS_LOG_STREAM=orders-logs vs platform-logs, set per service in the deploy pipeline. Consumers (Firehose transforms, Lambda) then get homogeneous schemas instead of mixed payloads from every app.

Custom formatter (new in v4.0.2). Add a formatter entry to the channel pointing at a class implementing Monolog\Formatter\FormatterInterface:

'kinesis' => [
    'driver'    => 'kinesis',
    'stream'    => env('KINESIS_LOG_STREAM'),
    'formatter' => App\Logging\KinesisFormatter::class,
    // ...
],

Use this to stamp every record with app, env, release, request_id/trace_id, and to normalise timestamps for Athena partitioning. Extend Monolog\Formatter\JsonFormatter and override normalize() rather than writing a formatter from scratch — you keep includeStacktraces, ignoreEmptyContextAndExtra, etc.

Assume-role / IAM-role deployments (new in v4.0.2). On ECS, EKS (IRSA), EC2, or Lambda, drop the credentials block entirely or set key/secret to null. The AWS SDK falls back to its default provider chain (env vars → shared ini → container/instance metadata → web identity token). Previously a null from env() would break the client construction, so this is the fix that lets you delete hard-coded credentials from config.

Log data, not strings. Log::info('order.paid', ['order_id' => $id, 'tenant' => $tenant, 'amount' => $amount]) — context lands in the record body and is queryable in Athena. Interpolating into the message gives you nothing to filter on.

Testing. In phpunit.xml set LOG_CHANNEL=null (or a stack without kinesis) so the suite never needs AWS. For the rare test that asserts emission, bind a mocked Aws\Kinesis\KinesisClient in the container and assert on putRecord/putRecords calls.

Infra pairing. The Laravel side is the easy half — provision shard count, retention, and the Firehose delivery stream in Terraform alongside it, and grant the task role kinesis:PutRecord + kinesis:PutRecords (plus DescribeStreamSummary/ListShards if the SDK path touches them).

Gotchas and Tips

• Buffering. The handler batches records and flushes on close()/destruct. In a long-running queue worker or Octane worker the buffer spans many jobs; if you need records out immediately (e.g. before a exit/dd), call Log::channel('kinesis')->close() to force the flush. Short-lived PHP-FPM requests flush naturally at shutdown.
• Kinesis PutRecords limits. 500 records and 5 MiB per request, 1 MiB per record. A fat exception trace with a giant context array will silently break the batch — truncate context (or cap includeStacktraces) in your formatter before you hit this in production.
• Throttling is invisible by default. Exceeding shard limits (1 MB/s or 1000 records/s per shard) returns ProvisionedThroughputExceededException per record inside a 200 response. If logs go missing, check CloudWatch WriteProvisionedThroughputExceeded before blaming Laravel.
• Credentials resolution order bit us before v4.0.2. Setting AWS_ACCESS_KEY_ID in .env but passing an empty string (not null) to credentials still produces a broken/partial credential object. Prefer omitting the key entirely over setting it to ''.
• env() and config caching. Never call env() inside the formatter or a custom handler — it returns null once php artisan config:cache has run. Read from config().
• Formatter output must be a single string per record. Returning an array or throwing from format() fails the whole batch, not just the offending record. Guard your normalisation.
• Local/dev without AWS. Point endpoint at LocalStack if the config exposes it; otherwise keep kinesis out of the local stack. There's no "no-op" fallback — an unreachable stream means dropped logs, not a crash, which makes it easy to miss until you need the logs.
• Debugging tip. Keep a second channel to single in the same stack while rolling this out. Compare the two streams to confirm nothing is being lost to throttling or batching, then drop single once the Firehose pipeline is proven.
• Extension points. If you need different flush semantics, subclass the package handler and point the channel's handler at it; or bypass the driver entirely with Log::extend('kinesis', fn ($app, $config) => ...) in a service provider for full control over the KinesisClient and handler stack.