Kafka Bundle Laravel Package

Getting Started

Know what you're adopting first. This is a 2-star bundle with zero dependents and a thin README — treat it as a small, readable piece of glue code rather than a supported framework component. The realistic cost of adoption is 30 minutes of reading src/ to confirm the API, not reading docs.

Minimal steps

composer require limlabs/kafka-bundle:1.0.2

Pin the exact version from the first composer require (see Gotchas). There is almost certainly no Flex recipe, so register it manually:

// config/bundles.php
LimLabs\KafkaBundle\LimLabsKafkaBundle::class => ['all' => true],

Then check the two files that define the real contract before writing any config:

• src/DependencyInjection/Configuration.php — the actual config tree keys and defaults.
• src/DependencyInjection/*Extension.php — which services get registered, and under what IDs, per config key.

Where to look first. Read, in order: the bundle class, the DI extension, then the producer/consumer classes. Whatever the README says, Configuration.php is the source of truth for config/packages/limlabs_kafka.yaml. Your first config should be the minimum that connects: broker list plus a default producer config. Add consumer groups only when you have a consumer.

First use case. Prove the round trip before designing anything: a console command that publishes one JSON message to a topic, and a second console command (or kafka:consume-style command if the bundle ships one) that polls the topic and dumps payloads. If you can't get that working against a local single-node Kafka or Redpanda container in an hour, that's your signal to stop and evaluate alternatives.

Run the consumer loop for longer than a single poll cycle — ten minutes idle-then-message, not thirty seconds — because the most recently fixed defect in this package (1.0.2) was recurring timeouts in exactly that path. A smoke test that only proves one message round-trips will not tell you whether the loop survives.

Prerequisites. Expect a dependency on ext-rdkafka (librdkafka) or a pure-PHP client — confirm from composer.json and pin your PHP/Symfony versions against the bundle's constraints, since a low-traffic package rarely keeps pace with new majors. Symfony 6.4/7.x compatibility should be verified explicitly rather than assumed.

Implementation Patterns

Wrap it before you use it. Given the maturity level, inject the bundle's producer into your own thin MessagePublisher interface rather than into controllers or domain services. One adapter class, one place to swap the implementation if the bundle stalls or you migrate to symfony/messenger with a Kafka transport.

Producer usage. The day-to-day shape is: inject the producer service from the container, call a send/publish method with topic, key, and payload, and let the bundle serialize. Always set a partition key when ordering matters for a given entity (e.g. order-{id}); leave it null only when order is genuinely irrelevant. Attach correlation/trace IDs as headers if the client supports them — that's what makes Kafka debuggable at 3am.

Consumers are workers, not requests. Whatever the bundle calls them, consumers belong in long-running console commands supervised by systemd or Supervisor. Structure your handler as a pure "handle one message" method and let the loop live in the command. That keeps the handler unit-testable and lets you change loop mechanics (batching, backoff, max-messages) without touching business logic.

Serialization at the edge. Keep the wire format dumb — JSON with an explicit type/version field — and deserialize into DTOs on consume. If the bundle offers Schema Registry/Avro support, verify it against a real registry before committing; that path is the most likely to be half-finished in a small package. If it doesn't, do the conversion yourself and don't fight the bundle.

Integration with the rest of Symfony. Publishing from inside a Doctrine transaction is a classic trap: publish after commit (or via an outbox table) so consumers never see messages for rows that rolled back. For consumers, use kernel.reset-style service resets or the bundle's own mechanism between messages to shed Doctrine identity-map and entity-manager state.

Testing. Unit-test handlers directly with a payload object. For the transport itself, run integration tests against Redpanda via Testcontainers rather than mocking the client — mocking a Kafka client mostly tests your mock. Keep a single docker-compose service for local dev so all developers share the same broker version. Add one slow test that leaves a consumer idle across several poll-timeout windows and then publishes; that is the shape of bug 1.0.2 fixed, and it's cheap insurance against a regression on your pinned version.

Deployment. One consumer group per logical consumer, and the group name must be stable across deploys. Add a liveness check (is the process polling? is consumer lag growing?) and a documented graceful-shutdown path so rebalances don't leave offsets dangling.

Gotchas and Tips

Upgrade to 1.0.2 before you debug timeouts yourself. The 1.0.2 changelog is a single line — "Fix recurring timeouts while consuming kafka messages" — which means 1.0.0/1.0.1 users experienced consumers stalling on a repeating cycle. If you are triaging a hung or periodically silent consumer on an older patch, bump the pin first; otherwise you will spend an afternoon tuning session.timeout.ms and max.poll.interval.ms against a bug that isn't yours. Note this is a one-line note, not a root-cause write-up: read the diff of the consumer/poll code between your version and 1.0.2 so you know what changed — in particular whether the bundle now swallows timeouts internally rather than propagating them. If it does, any shutdown logic you built on "the poll throws on timeout" stops firing, and an idle-looking consumer may no longer be distinguishable from a wedged one except by your own liveness metric.

Docs will be stale; source won't. Expect config key names, class names, and command signatures in the README to be wrong or incomplete. Diff the README against Configuration.php on day one and write a short internal note with the verified API. That note will outlive the README. The changelog is equally thin — it describes symptoms, not APIs, so it is not a substitute for reading the diff.

ext-rdkafka installation is the real onboarding cost. Build problems come from missing librdkafka-dev, mismatched librdkafka versions between image and extension, and musl-vs-glibc base images. Pin the extension version in your Dockerfile and build from the same base image everywhere. A mismatch here produces cryptic errors that look like bundle bugs — including timeout-shaped ones, which is why you want the extension pinned and the bundle on 1.0.2 before blaming either.

Offset semantics decide your reliability story. Check whether the bundle commits automatically or exposes manual commit. Auto-commit gives you at-most-once-ish behaviour and silent message loss on crash; manual commit gives at-least-once and obligates you to make handlers idempotent. Assume at-least-once and design for duplicate delivery — dedupe on a message ID. Timeout handling interacts with this: a poll timeout that triggers a rebalance can re-deliver messages from the last committed offset, so a timeout fix does not remove the need for idempotency.

Consumer loops and Symfony's lifecycle. The kernel is not designed to run for days. Watch for memory growth from Doctrine, logging handlers, and accumulated data in the profiler. Prefer a --max-messages or time limit plus a supervisor restart, and disable the profiler and debug consumers in worker environments.

Poll timeouts and EOF. If consumers appear to hang, check poll timeout settings and whether partition-EOF events are treated as errors. A consumer that only logs on exception will look dead when it's just idle. Log the timeout path explicitly — at debug level if it's now expected — so that after the 1.0.2 fix you can still tell "polling fine, no messages" from "polling stuck".

Logging and observability are on you. The bundle almost certainly gives you little beyond exception propagation — and a timeout fix that suppresses recurring noise may reduce your visible signal further. Emit structured logs per message (topic, partition, offset, key, duration), and export consumer lag to your metrics system. Without lag metrics you cannot tell "slow" from "broken", and you cannot tell whether 1.0.2 actually resolved your stall or merely stopped printing about it.

Extension points are your escape hatch. Because the package is small, override services in services.yaml (by ID, after the bundle's extension runs) rather than forking. If you must fork, pin an exact version and vendor the fork under a new name so Composer updates can't silently change consumer semantics — doubly so here, where the consumer loop behaviour changed in a patch release.

Version pinning is non-negotiable. Use an exact constraint (1.0.2, not ^1.0) and bump deliberately after reading the diff: 1.0.2 is a patch that changes message-consumption behaviour, which is precisely the kind of change a caret range would have handed you unreviewed in production. Re-evaluate the dependency each time you upgrade Symfony. Keep the adapter boundary from the Implementation section so the migration off this bundle — to symfony/messenger's Kafka transport or a more established client — is a one-file change.