Kafka Bundle Laravel Package

Getting Started

First, the reality check. m6web/kafka-bundle is archived and its last release was June 2017. It is a Symfony bundle, not a Laravel package: it ships a Bundle subclass, DI extension, and YAML config tree, and it depends on ext-rdkafka (the PHP binding over librdkafka). In a Laravel app there is no bundle kernel to register it with, so "day-to-day leverage" here means either (a) reading it as a reference implementation while wiring ext-rdkafka yourself, or (b) lifting its service wiring into a Laravel service provider. Do not expect composer require + auto-discovery to work.

Minimal steps to evaluate it:

1. composer require m6web/kafka-bundle in a throwaway Symfony 3.x skeleton only — installing it into a modern Laravel app will fight you on PHP/Symfony constraints.
2. Install the real dependency first: pecl install rdkafka (or your distro package), confirm php -m | grep rdkafka.
3. Read in this order: README.md → DependencyInjection/Configuration.php (the actual config contract — it is more authoritative than the README) → Resources/config/services.yml (service IDs you'd inject) → Command/ (the long-running consumer command).
4. Prove the loop end-to-end with a single local broker (Redpanda or bitnami/kafka via Docker): produce one JSON message from a console command, consume it in a second terminal.

The canonical first use case: produce from a web request, consume from a CLI worker. The bundle's value is exactly that split — a producer service you inject and call, and a supervised consumer command that owns the poll loop. It gives you almost nothing else: no schema registry, no serialization contract, no retry/DLQ machinery.

Implementation Patterns

Producer: inject, don't instantiate. The bundle registers a producer service (confirm the exact ID in Resources/config/services.yml; the config root looks roughly like brokers + per-topic producer settings). The pattern that survives translation to Laravel:

// Illustrative shape — verify service id against the repo.
public function __construct(private Producer $producer) {}

public function publish(array $event): void
{
    $this->producer->send(
        'orders.created',                 // topic
        json_encode($event, JSON_THROW_ON_ERROR),
        $event['order_id']                // key => partition affinity
    );
    $this->producer->flush(2000);         // ms; async by default
}

In Laravel, mirror this with a singleton bound in a service provider, wrapping RdKafka\Producer. Register it as a singleton (the connection is expensive) and flush() in a terminating callback or register_shutdown_function.

Consumer: a supervised CLI command, never a request. The bundle's consumer command is the model: a while (true) poll loop, RD_KAFKA_RESP_ERR__PARTITION_EOF treated as "no more messages right now, sleep briefly", and a consumer group ID so partitions rebalance across N worker processes. Run it under supervisor/systemd, one process per desired parallelism, and never inside PHP-FPM.

Offset discipline. Set enable.auto.commit=false and commit after your side effect succeeds. Kafka gives at-least-once; your handler must be idempotent (upsert by event ID, or a processed_events table). This is the single decision that determines whether the system is safe, and the bundle makes it for you only if you configure it.

Topic keying. Always set a message key when ordering matters — same key, same partition, guaranteed order. No key means round-robin and interleaved updates for the same entity.

Integration tips for Laravel specifically:

• Keep the Kafka contract (KafkaProducer::send()) as your own interface; put the rdkafka call behind it. That way the archived bundle is a detail you can swap for mateusjunges/laravel-kafka or enqueue/rdkafka without touching call sites.
• Separate config: config/kafka.php with brokers, group IDs, topic names. Do not hardcode topic strings in handlers.
• Log with correlation IDs carried in the message envelope ({id, type, version, occurred_at, payload}), because Kafka has no request context to inherit.
• Test the producer by swapping the interface for a fake; test the consumer with a real broker in CI, not a mock — rebalance and offset bugs only appear against real Kafka.

Gotchas and Tips

The deprecation is the headline gotcha. Last release 2017, 8 stars, zero dependents. That means: no PHP 8 support, no Symfony 5/6, no typed properties, no maintenance, and no community to answer questions. librdkafka has moved a long way since; the extension API surface it wraps has drifted. Treat it as documentation of a design, not a dependency.

Bundle ≠ package. In Laravel you cannot registerBundles(). If you insist on using it, you'll be manually constructing services that expect Symfony\Component\DependencyInjection\ContainerInterface and EventDispatcherInterface. The symfony/* transitive requirements will also bloat your vendor/ and can conflict with Laravel's own constraints.

Config quirks. With Symfony's config cache, changes to the bundle's YAML require a cache clear; the analogous Laravel failure is config:cache silently freezing stale broker lists. When a config change "doesn't apply", check the cache before the code.

Long-running consumers leak. PHP is not built for week-long processes. Watch RSS, call gc_collect_cycles() periodically, and recycle workers (supervisor stopwaitsecs, or a max-message counter that exits cleanly after N messages). Also raise memory_limit and be explicit about it in the supervisor config, not the CLI default.

Rebalance storms. Default session.timeout.ms / max.poll.interval.ms are unforgiving: a slow handler causes the consumer to be kicked from the group, offsets not to commit, and the same batch reprocessed forever. Either speed up batches or shorten max.poll.records.

Producer buffering hides failures. send() is asynchronous. If you never poll() or flush(), delivery-report errors are invisible and messages vanish on process exit. In short-lived web requests, flush() with a timeout and check the return code — otherwise a broker outage looks like a successful HTTP 200.

Message size. message.max.bytes on the broker and queue.buffering.max.messages on the producer both bite quietly. Large payloads should be a pointer (S3/DB row + ID), not the blob.

Extension points (if you do keep it): override the producer/consumer service definitions in your container, add a Symfony event listener around the consumer to hook metrics, and subclass the consumer command for custom offset-commit logic. In a Laravel port, the equivalents are container bindings, a middleware-style decorator around the handler, and a Command subclass.

Migration mapping, since you will migrate:

If the requirement is "Kafka-backed work queue with per-job retries and failure tracking", stop and reconsider: Kafka is a partitioned log, not a job queue. Laravel's queue + Redis/SQS may be the correct answer, and the archived bundle is a strong signal that the previous team's answer aged out.