Kafka Bundle Laravel Package

Getting Started

sts-gaming-group/kafka-bundle is a Symfony bundle (not a Laravel package), so treat it as a library-plus-config-tree you wire into Laravel yourself rather than something that auto-registers.

1. Check the platform requirements first. The bundle sits on top of ext-rdkafka (php-rdkafka) and flix-tech/avro-serde-php. Confirm librdkafka and the PHP extension versions on your host before composer require — a mismatch here is the single most common install failure, and it surfaces as a fatal at boot, not at install.

   php -m | grep rdkafka        # expect: rdkafka
   php --ri rdkafka | grep -i version
   composer require sts-gaming-group/kafka-bundle:^1.5
   

2. Read the source, not the README. The README is thin. Open these three files in order:
   • composer.json — confirms the ext-rdkafka constraint and, as of 1.5.1, the flix-tech/avro-serde-php: ^3.0 range.
   • DependencyInjection/Configuration.php — this is the authoritative list of config keys (brokers, topic names, schema-registry URL, serializer settings, consumer group IDs). Your Laravel config/kafka.php should mirror this tree exactly.
   • DependencyInjection/*Extension.php — shows which services are registered, their IDs, and any container tags. You need this to know what to bind() in your service provider.
3. First use case: publish one Avro-encoded event. Get a single producer working end-to-end (encode → send → read back with kafka-console-consumer) before touching consumers. Producers are stateless and easy to debug; a broken consumer loop will hide serialization bugs behind offset commits.

Because the bundle is not Laravel-native, expect to write a ~20-line service provider that loads the config array and registers the producer/consumer services manually. Budget for that; it is the real "getting started" cost.

Implementation Patterns

Config bridge. Point the bundle's config at a plain Laravel config array and translate env vars in one place:

// config/kafka.php
return [
    'brokers'        => env('KAFKA_BROKERS', '127.0.0.1:9092'),
    'schema_registry'=> env('KAFKA_SCHEMA_REGISTRY'),
    'consumer_group' => env('KAFKA_GROUP', config('app.name')),
    'topics'         => [ 'orders.placed' => App\Events\OrderPlaced::class ],
];

Keep this file 1:1 with the bundle's Configuration.php node names — every rename you invent is a translation layer you have to maintain.

Producer as a singleton. Register the producer in AppServiceProvider::register() as a singleton and inject it into a small EventPublisher wrapper. This is the key integration tip: never let bundle service IDs leak into your domain code. Your EventPublisher interface means you can swap the transport for a sync/log driver in tests and local dev without the bundle installed.

$this->app->singleton(EventPublisher::class, fn ($app) =>
    new KafkaEventPublisher($app->make('kafka.producer'), config('kafka'))
);

Consumer as an Artisan command. The bundle's consumer is a long-running loop; wrap it in an artisan kafka:consume {topic} command and run it under Supervisor. Use --once/max-messages flags for local dev and CI so tests never hang.

One handler per topic. Map topic → handler class in config (as above) and dispatch through the Laravel container. This gives you queue-worker-style ergonomics and makes Event::fake()-style assertions possible.

Schema-first workflow. Define .avsc files in resources/avro/, register them in CI against a throwaway schema registry (docker-compose), and only then deploy producers. This is the pattern that actually prevents production incidents — the registry is the contract, not the PHP classes.

Test seam. Assert on the EventPublisher interface, not the producer. For integration tests, run a single-broker KRaft container plus a schema registry in CI and gate on KAFKA_INTEGRATION=1.

Gotchas and Tips

• The 1.5.1 change is the one to audit. It widens flix-tech/avro-serde-php from ^2 to ^3.0. That is not a breaking change in this bundle, but it silently drops you onto Avro serde v3 if your lockfile resolves that way. v3 brings its own serializer API and subject-name-strategy behavior — run your schema-registry integration tests and diff the wire format for one topic before shipping. If you need determinism, pin flix-tech/avro-serde-php explicitly in your root composer.json rather than inheriting the bundle's range.
• Symfony bundle ≠ Laravel package. No package discovery, no config/ publishing, no vendor:publish. Container tags (e.g. handler or subscriber tags) are not honored by Laravel's container — you must resolve tagged services manually or register them by hand.
• Long-running consumers leak Laravel state. The container, DB connections, and any per-request singletons persist across messages. Reconnect the DB on LostConnection, avoid accumulating Eloquent models in memory, and restart workers on deploy. pcntl signal handling must be registered by you — the Symfony bundle's graceful-shutdown wiring won't run in an Artisan command unless you replicate it.
• Offset semantics are at-least-once. Handlers must be idempotent (upsert by event ID, not blind inserts), and you should commit after the handler succeeds. If you commit before, a handler crash silently drops the message.
• Consumer group rebalances will re-deliver in-flight messages. Set session.timeout.ms and max.poll.interval.ms deliberately — a slow handler that exceeds the poll interval gets evicted and re-processes its whole batch.
• Schema compatibility mode. Default is usually BACKWARD; removing or renaming a field without a default will be rejected at registration, not at publish time. Validate .avsc changes in CI, not in production.
• Debugging tip: when a message fails to decode, dump the raw bytes before the deserializer and check the magic byte (0x00) and 4-byte schema ID prefix. A payload without that prefix is either plain JSON or was produced by a different serializer — a very common mismatch when mixing producers.
• Extension points are limited. The bundle offers little in the way of pluggable serializers or middleware; if you need custom headers, interceptors, or a DLQ, expect to wrap the producer/consumer services rather than extend them. That wrapper is also your upgrade insulation.

Bottom line on 1.5.1: low-risk dependency widening, no breaking changes, no deprecations. Upgrade opportunistically, but pin flix-tech/avro-serde-php in your own composer.json and re-run schema-registry integration tests — that is where the real variance lives.