Getting Started

Know what you're adopting first. koco/messenger-kafka is a symfony/messenger transport, not a Illuminate\Queue driver. It implements TransportInterface (sender + receiver + ack/reject), so it will not work with php artisan queue:work, Horizon, or ShouldQueue out of the box. If you're a Laravel dev, you have three realistic entry points:

Produce from Laravel, consume in a Symfony service. Most common and lowest risk — run this package in the consumer service only.
Produce from Laravel by wiring Symfony Messenger into the app. Laravel already ships several Symfony components; adding symfony/messenger + symfony/event-dispatcher is doable but you own the wiring.
Adapt it to a Laravel queue driver. Technically possible (wrap the TransportInterface in a Illuminate\Contracts\Queue\Queue), but you'll be writing more glue than the package saves you — look at mateusjunges/laravel-kafka first.

Check platform fit before you install (changed in v0.19). v0.19 is titled "Support maintained Symfony versions" — it drops EOL Symfony releases and constrains the package to the currently maintained line (6.4 LTS and 7.x). Practical consequences:

On Laravel 10 you get Symfony 6.3/6.4 components; Laravel 11/12 give you 7.x. Older Laravel (9 and below) pins Symfony 5.4/6.0–6.3 and will most likely conflict — you'd stay on 0.18.x or vendor the transport.
Read composer.json for the exact symfony/messenger constraint and .github/workflows/* for the tested PHP/Symfony matrix. v0.19 also migrated CI from the previous provider to GitHub Actions, so the matrix is now explicit and public — that's the fastest way to know which combination is actually exercised versus merely allowed by the constraint.
If you're already on 0.18 with a modern stack, v0.19 lists no API changes: expect a drop-in bump, but still re-run your transport integration tests (dependency bumps can shift serializer/worker behaviour).

Prerequisites

ext-rdkafka (pecl, backed by librdkafka). It is not installable by Composer alone.
```
RUN pecl install rdkafka && docker-php-ext-enable rdkafka
```
symfony/messenger, plus symfony/serializer if you want JSON on the wire.

composer require koco/messenger-kafka:0.19.* symfony/messenger symfony/event-dispatcher symfony/serializer

Minimal transport config. The package follows the standard Messenger factory contract: DSN for brokers, an options array for the topic plus any librdkafka properties. Anything not recognised by the package is forwarded to RdKafka\Conf, so group.id, security.protocol, sasl.*, auto.offset.reset, enable.auto.commit, etc. are all valid keys.

# config/messenger.yaml (load it yourself in Laravel; see below)
transports:
  kafka_events:
    dsn: '%env(KAFKA_DSN)%'            # kafka://broker1:9092,broker2:9092
    serializer: messenger.transport.symfony_serializer
    options:
      topic: 'app.events'
      group.id: 'laravel-consumers'
      auto.offset.reset: 'earliest'

The transport option used for the consumer group has shifted between releases (group.id vs a dedicated consumer key). Check the README table for the tag you pin — this is a small, fast-moving package, and pre-1.0 minor bumps routinely rename options.

Where to look first in the source (it's only a few hundred lines):

composer.json + .github/workflows/ — supported Symfony range and the tested matrix.
KafkaTransportFactory — DSN parsing and option mapping.
KafkaTransport / KafkaReceiver / KafkaSender — where offsets are committed, where headers become librdkafka headers.
KafkaTransport::ack() / reject() — this is where your at-least-once semantics live. Read it before you trust it in production.

First use case: publish domain events (e.g. OrderPlaced, UserRegistered) to a topic from Laravel, and consume them in a service that already runs Symfony Messenger — analytics, search indexing, notifications. This is the path of least resistance and sidesteps the Laravel/Messenger impedance mismatch.

Implementation Patterns

1. Producer: bridge Laravel events to Messenger

Keep Messenger's bus out of your controllers. Bind the transport as a singleton and forward from a listener.

// app/Providers/KafkaServiceProvider.php
use Koco\Kafka\Messenger\KafkaTransportFactory;
use Symfony\Component\Messenger\Transport\Sender\SenderInterface;
use Symfony\Component\Messenger\Transport\Serialization\SerializerInterface;

public function register(): void
{
    $this->app->singleton(SenderInterface::class, function ($app) {
        return (new KafkaTransportFactory())->createTransport(
            config('services.kafka.dsn'),
            config('services.kafka.options'),
            $app->make(SerializerInterface::class),
        );
    });
}

// app/Listeners/PublishOrderPlaced.php
public function handle(OrderPlaced $event): void
{
    $this->sender->send(new Envelope($event));
}

Two things to get right here:

Don't produce inside the DB transaction. Dual writes (MySQL + Kafka) are inconsistent by construction. Use a transactional outbox table and a relay command, or accept fire-and-forget semantics explicitly.
Flush the producer. librdkafka buffers; if the PHP process exits before flush(), the message is gone. In a web request, a synchronous flush adds latency — consider linger.ms on the producer config and a flush with a bounded timeout.

2. Consumer: a real artisan command around Messenger's `Worker`

There's no queue:work equivalent. Wrap Symfony\Component\Messenger\Worker yourself:

protected $signature = 'kafka:consume {--time-limit=3600} {--memory-limit=128M}';

public function handle(ReceiverInterface $receiver, MessageBusInterface $bus): int
{
    $dispatcher = new EventDispatcher();
    $dispatcher->addSubscriber(new StopWorkerOnTimeLimitListener((int) $this->option('time-limit')));
    $dispatcher->addSubscriber(new StopWorkerOnMemoryLimitListener(...));
    $dispatcher->addSubscriber(new StopWorkerOnSigtermSignalListener());

    (new Worker([$receiver], $bus, $dispatcher))->run();

    return self::SUCCESS;
}

On the Symfony versions v0.19 targets (6.4 / 7.x) all of these listeners are available without shims, and Worker's dispatcher argument is nullable — the above is the supported shape. If you're pinned to 0.18 on an older Symfony, check which listeners exist in your version before copying this.

Run the command under Supervisor (or your orchestrator) exactly as you would queue:work. Always set a time limit — PHP workers accumulate memory and librdkafka handles are long-lived.

3. Message design for a polyglot topic

Plain DTOs, no Laravel models. Kafka messages outlive your schema. Use readonly value objects with scalar fields and a version property.
JSON serializer, not the native PHP one. messenger.transport.native_php_serializer is tempting for same-stack, but it makes the topic unreadable to any other consumer (and any other language). Use the Symfony serializer with the JSON encoder.
Stamp headers leak. The Symfony serializer writes X-Message-Stamp-* headers. Non-Symfony consumers must ignore them; if you control both sides, consider a tiny custom SerializerInterface that emits {"type": "...", "payload": {...}} with a single type header — far easier to consume from Node/Python.
Carry an idempotency key (event_id, order_id) in the payload. Kafka gives you at-least-once, and Messenger retries re-publish to the same topic, so duplicates are a design assumption, not an edge case.

4. Ordering and scaling

Ordering is per-partition only. If order matters, set the message key (customer ID, aggregate ID) so related messages land on the same partition — check how the transport exposes this (transport option or header) in your pinned version. Then size your consumer group so partition count ≥ worker count; extra workers sit idle by design.

5. Retries and failure transport

Use Messenger's RetryStrategy and a failure_transport (Doctrine or Redis — do not point the failure transport back at Kafka, or you get an infinite loop of a message that always fails). Remember that a retry re-publishes, which means:

the retried message gets a new partition assignment (ordering broken for that aggregate);
a crashed consumer after ack() but before the commit window can replay the batch.

Prefer: validate cheaply, fail fast on poison messages, route to the failure transport after N attempts, and alert on its depth.

6. Hybrid setup (most likely for a Laravel shop)

Laravel (mateusjunges/laravel-kafka or raw rdkafka producer)
        │  JSON + headers
        ▼
   Kafka topic  ◄── koco/messenger-kafka (Symfony consumer, or a Laravel app running Messenger)
        │
        ▼
  Analytics / notifications / search indexing

This lets the Laravel side stay idiomatic while the consumer side gets Messenger's middleware stack (retry, validation, logging). It also decouples you from this package's Symfony-version constraints: only the consumer image needs a compatible Symfony.

7. Testing

Handlers: unit test with a fake bus, no Kafka involved.
Transport: Symfony\Component\Messenger\Transport\InMemoryTransport for end-to-end bus tests.
Integration: Redpanda or Kafka in KRaft mode via docker compose in CI — mocking RdKafka\KafkaConsumer is not worth the effort. Mirror the Symfony/PHP pair the package's own GitHub Actions matrix uses; that's the combination with upstream coverage.

Gotchas and Tips

Packaging and maturity

Pre-1.0 (0.x) with a single maintainer. Semver gives no stability guarantee on minor bumps — pin an exact version (0.19.0, not ^0.19) and be prepared to vendor/fork. The surface you depend on is small (a factory, a transport, a receiver, a sender), so forking is genuinely viable.
v0.19 is a maintenance release: Symfony constraint update + CI moved to GitHub Actions. Treat it as a governance signal as much as a code change — the new workflow matrix is the authoritative list of supported PHP/Symfony pairs, and it's the file to read before you file a bug. Older Symfony lines are now unsupported; if you need them, stay on 0.18 and own the patches.
Confirm the package name and repo home line up with what you're installing — the release notes point at KonstantinCodes/messenger-kafka, while the Composer vendor is koco. Verify on Packagist that the tag you pin belongs to the same repository you're reading, and don't trust release-date metadata from third-party mirrors; check the git tag itself.
Alternatives worth benchmarking: enqueue/messenger-adapter + enqueue/rdkafka (heavier, more abstracted), mateusjunges/laravel-kafka (native Laravel ergonomics), or a thin custom TransportInterface if your needs are narrow.

Dependency resolution with Laravel

Laravel pins symfony/* transitively via laravel/framework. Requiring symfony/messenger at a version outside your framework's range produces a long, confusing conflict report that is really just "your Laravel is too old for this transport". Check composer why symfony/messenger before blaming the package.
Don't install symfony/messenger in a Laravel app and also pull in a full Symfony skeleton; you'll get duplicated service definitions and a second event_dispatcher. Take the components, not the framework.

The extension is the real dependency

ext-rdkafka needs librdkafka ≥ 1.9 for modern broker compatibility (KRaft, SASL/SCRAM). Pin the extension version in your base image and keep it consistent between producer and consumer images — mismatched librdkafka versions produce wonderfully opaque rebalance behaviour.
rdkafka is not fork-safe. Don't share a producer or consumer across pcntl_fork, Swoole coroutines, Octane workers, or spatie/laravel-fork. Create per-process handles. If you're on Octane, produce from a queued job or a dedicated worker, not from a request-scoped singleton.

Offset and commit semantics

Verify that enable.auto.commit and enable.auto.offset.store are set to false by the transport. If they're left at librdkafka defaults, offsets commit on a timer regardless of handler success, and you silently lose messages on failure.
commit_async (if exposed in your version) trades durability for throughput: a crash between handling and commit replays messages. Fine for idempotent handlers, wrong for payments.
Long handlers can exceed max.poll.interval.ms (default 5 min), get kicked from the group, and trigger a rebalance that replays in-flight messages. Either lower the batch size, raise the interval, or heartbeat from a supervisor loop.

Config quirks

librdkafka keys contain dots (group.id, sasl.mechanisms). Quote them in YAML; in Laravel config/services.php they're just array keys, which is safer.
Never build Messenger config inside a closure if you cache config — Laravel's config:cache serialisation will break it. Use plain arrays.
DSNs with multiple brokers are comma-separated; URL-encode SASL credentials (@, :, / in passwords will silently truncate the DSN otherwise).
auto.offset.reset=latest is the librdkafka default and a classic "why is my consumer idle?" — set earliest in dev.
Pre-create topics with the right partition count. Auto-creation usually yields 1 partition, which caps your consumer group at 1 active worker forever.

Operational

librdkafka logs to stderr by default and can spam. Set log_level and debug deliberately; wire the transport's error callback into Laravel's logger if your version supports one.
Graceful shutdown matters more than with Redis queues: on SIGTERM, stop the worker, let in-flight messages finish, flush() the producer with a timeout. Otherwise you get duplicates on every deploy.
Monitor consumer lag (kafka-consumer-groups --describe) as your primary health signal, not "is the worker running".
Message size: default message.max.bytes is 1 MB. Serialized envelopes with stamps blow past that faster than you'd expect; check the broker's message.max.bytes and the topic's max.message.bytes together.
Kafka has no per-message TTL or dead-letter concept. Retention is a topic property — set it deliberately, or a stuck consumer group will replay a week of events when you finally fix it.

Messenger Kafka Laravel Package