rybakit/msgpack
Pure PHP implementation of the MessagePack serialization format, fully compliant with the latest spec. Supports streaming unpacking, unsigned 64-bit integers, custom object types and extensions, with good performance and full test coverage.
Install with Composer. There is no Laravel service provider to publish:
composer require rybakit/msgpack
Start with the README sections Packing, Unpacking, and Custom types, then look at src/Type, src/TypeTransformer, and examples/MessagePack for real extension examples.
The simplest first use case is packing an array or DTO into Redis or a cache store:
use MessagePack\Packer;
use MessagePack\BufferUnpacker;
$packer = new Packer();
$packed = $packer->pack(['id' => 1, 'name' => 'Ada', 'roles' => ['admin']]);
Redis::set('user:1:profile', $packed);
$unpacker = new BufferUnpacker();
$profile = $unpacker->reset(Redis::get('user:1:profile'))->unpack();
For one-off calls you can use the static facade-like API:
use MessagePack\MessagePack;
$packed = MessagePack::pack($value);
$value = MessagePack::unpack($packed);
In a Laravel app, register configured instances once so every producer and consumer uses the same options:
// config/msgpack.php
use MessagePack\PackOptions;
use MessagePack\UnpackOptions;
return [
'pack_options' => PackOptions::DETECT_ARR_MAP
| PackOptions::DETECT_STR_BIN
| PackOptions::FORCE_FLOAT64,
'unpack_options' => UnpackOptions::BIGINT_AS_STR,
];
// App\Providers\AppServiceProvider::register()
use MessagePack\Packer;
use MessagePack\BufferUnpacker;
$this->app->singleton(Packer::class, fn () => new Packer(
config('msgpack.pack_options')
));
$this->app->bind(BufferUnpacker::class, fn () => new BufferUnpacker(
'',
config('msgpack.unpack_options')
));
Packer is effectively stateless and can be a singleton. BufferUnpacker is stateful; bind it, do not share one instance across requests or long-running workers.
Wrap serialization behind a small repository/service. Avoid sprinkling new Packer() and raw unpack() calls throughout controllers. A dedicated class gives you one place to version payloads, enforce size limits, and swap options:
final class MessagePackCodec
{
public function __construct(
private readonly Packer $packer,
private readonly BufferUnpacker $unpacker,
) {}
public function encode(mixed $value): string
{
return $this->packer->pack($value);
}
public function decode(string $payload): mixed
{
return $this->unpacker->reset($payload)->unpack();
}
}
Use an explicit envelope for stored payloads. MessagePack itself has no schema version. Store a small header or a version key alongside the blob so you can migrate later:
$envelope = $packer->pack([
'v' => 2,
'type' => 'user.profile',
'data' => $profile,
]);
This is especially important for Redis keys, database blobs, and queue messages that outlive a deploy.
Model casts for binary database columns. MessagePack is a good fit for flexible JSON-like attributes when you want less size and faster parsing:
use Illuminate\Contracts\Database\Eloquent\CastsAttributes;
use MessagePack\Packer;
use MessagePack\BufferUnpacker;
final class AsMessagePack implements CastsAttributes
{
public function get($model, string $key, $value, array $attributes): mixed
{
return $value === null
? null
: app(BufferUnpacker::class)->reset($value)->unpack();
}
public function set($model, string $key, $value, array $attributes): ?string
{
return $value === null
? null
: app(Packer::class)->pack($value);
}
}
Use a LONGBLOB on MySQL, bytea on PostgreSQL, or BLOB on SQLite. Do not store raw packed bytes in a UTF-8 text column.
HTTP APIs and content negotiation. A middleware can decode application/x-msgpack request bodies, and a response macro can encode responses:
// Middleware: decode request
if ($request->header('Content-Type') === 'application/x-msgpack') {
$data = app(BufferUnpacker::class)
->reset($request->getContent())
->unpack();
$request->merge(is_array($data) ? $data : ['payload' => $data]);
}
// Response
return response($packer->pack($data), 200, [
'Content-Type' => 'application/x-msgpack',
]);
When calling other services:
Http::withBody($packer->pack($payload), 'application/x-msgpack')
->post('https://service.internal/events');
Streaming and chunked input. This is where BufferUnpacker is more than a static helper. Use append() with tryUnpack() when reading from TCP sockets, Redis streams, or chunked files:
$unpacker = app(BufferUnpacker::class);
$unpacker->reset('');
foreach ($chunks as $chunk) {
$unpacker->append($chunk);
foreach ($unpacker->tryUnpack() as $message) {
$this->handle($message);
}
// Drop already-consumed bytes; keeps unread partial messages.
$unpacker->release();
}
unpack() throws InsufficientDataException if the current buffer does not contain a complete value. tryUnpack() returns an array of complete messages and avoids that exception, which is the right choice for stream loops.
Custom types: choose the lightest mechanism.
CanBePacked on your own value objects when you control the class.CanPack transformer for classes you do not own, resources, or framework types.Extension only when you need both pack and unpack support, or when interoperating with other MessagePack implementations via extension type IDs.use MessagePack\Packer;
use MessagePack\TypeTransformer\StreamTransformer;
$packer = new Packer(null, [new StreamTransformer()]);
$packed = $packer->pack(fopen('/path/to/file', 'rb'));
Laravel queues and events. Laravel already serializes jobs and events internally; do not try to replace the queue serializer. Instead, pack large or cross-language sub-payloads inside a job property or event payload:
final class ImportChunk implements ShouldQueue
{
public function __construct(
public readonly string $packedRows,
) {}
}
dispatch(new ImportChunk($packer->pack($rows)));
This is useful when the consumer is a non-PHP worker, or when you want to move a large array through Redis without PHP's serialize() overhead.
Timestamp handling. The Timestamp extension is registered by default only on the static MessagePack::pack() / MessagePack::unpack() helpers. With your own Packer and BufferUnpacker, register it explicitly and remember that extendWith() returns a new instance:
use MessagePack\Extension\TimestampExtension;
use MessagePack\Type\Timestamp;
$timestampExtension = new TimestampExtension();
$packer = (new Packer())->extendWith($timestampExtension);
$unpacker = (new BufferUnpacker())->extendWith($timestampExtension);
$packed = $packer->pack(Timestamp::now());
$timestamp = $unpacker->reset($packed)->unpack();
Per-boundary configuration. Do not use one global option set blindly for every integration. Internal PHP-to-PHP traffic can use FORCE_BIN, FORCE_MAP, and FORCE_FLOAT64 for speed. Public or cross-language APIs should usually use DETECT_STR_BIN, DETECT_ARR_MAP, and BIGINT_AS_STR, because those are the safest defaults for heterogeneous consumers.
Testing. Keep fixtures as hex strings so failures are readable, and round-trip test each boundary:
$this->assertSame(
bin2hex($expectedPacked),
bin2hex($packer->pack($value))
);
Add at least one cross-language fixture if you exchange data with JavaScript, Go, Python, or Rust.
PHP arrays do not map cleanly to MessagePack arrays and maps. By default, sequential numeric keys become a MessagePack array; anything else becomes a map. Empty arrays are ambiguous. Use packMap() or new Map([...]) when the consumer requires a map:
$packer->packMap([1, 2]); // {0: 1, 1: 2}
$packer->pack(new Map([1, 2])); // {0: 1, 1: 2}
Map keys are coerced by PHP on unpack. MessagePack maps can contain non-string keys, but PHP arrays only support int and string keys. Numeric string keys become integers. If exact key types matter, unpack into a DTO or transform before storage.
FORCE_STR can corrupt binary data. Encrypted payloads, images, compressed blobs, and random bytes are not valid UTF-8. Packing them as MessagePack str is wrong. Use FORCE_BIN, DETECT_STR_BIN, or explicitly wrap the value in Bin. Conversely, if you know every string is UTF-8, FORCE_STR avoids detection overhead.
DETECT_STR_BIN is not free. It performs UTF-8 validation on strings, which is noticeable on large payloads. If a boundary is guaranteed UTF-8 text, force str; if it is guaranteed binary, force bin.
FORCE_FLOAT32 loses precision. Use it only when the consumer expects 32-bit floats or you have measured that the precision loss is acceptable. The default FORCE_FLOAT64 is the safe choice.
Unsigned 64-bit integers overflow PHP integers. The MessagePack spec allows values up to 18446744073709551615, which cannot fit in a PHP int. The default BIGINT_AS_STR avoids data loss. BIGINT_AS_GMP and BIGINT_AS_DEC require the GMP or Decimal extension. If you exchange IDs with JavaScript, remember that JS Number also cannot represent all 64-bit integers safely; send IDs as strings or use BigInt in the consumer.
extendWith() is immutable. This is a common mistake:
$packer->extendWith(new TimestampExtension()); // result discarded
$packer = $packer->extendWith(new TimestampExtension()); // correct
Register extensions on both the packer and unpacker. Packing a timestamp without the unpacker extension means the consumer receives a raw Ext object or throws, depending on context.
Extension IDs are a protocol contract. Application-specific extension IDs are integers from 0 to 127. Do not reuse IDs for different types, and do not change the ID after data is stored. Add an application-level schema version if extension payloads evolve.
Keep BufferUnpacker request-scoped. It retains buffer state. A singleton unpacker in a queue worker or Octane process will leak state between messages. Always call reset() before decoding a fresh buffer, or resolve a new instance.
Check for trailing bytes when decoding a single payload. unpack() returns the first value and leaves the rest in the buffer. If you expect exactly one message, assert:
$value = $unpacker->reset($payload)->unpack();
if ($unpacker->hasRemaining()) {
throw new RuntimeException('Unexpected trailing MessagePack data');
}
Do not unpack untrusted payloads without limits. MessagePack length prefixes can describe very large arrays, maps, strings, and binary blobs. A malicious payload can trigger large allocations before you ever see the value. Enforce HTTP body size limits at the web server or middleware, authenticate internal traffic, and prefer rejecting oversized payloads before decoding.
There is no automatic PHP object hydration. MessagePack is not PHP serialize(). Packing an arbitrary object will fail unless it implements CanBePacked or a transformer is registered. This is good for security, but it means you should pack arrays, DTOs, or explicitly transformed objects, not Eloquent models directly. Convert models with toArray() or a dedicated DTO first.
Do not confuse this with phpredis' built-in MessagePack serializer. Redis::SERIALIZER_MSGPACK in phpredis is a C extension feature. rybakit/msgpack is pure PHP and does not plug into Laravel's cache serializer by default. You are responsible for calling pack() and unpack() at the storage boundary.
Prefer type-specific unpack methods at trusted boundaries. unpackMap(), unpackArray(), unpackStr(), and unpackBin() fail loudly if the payload does not match the expected type. This is useful for protocol validation.
Use seek(), skip(), read(), and getRemainingCount() for framed protocols. If you have a length-prefixed envelope, read the header, skip or seek to the payload, and decode only what you need. seek(-N) is handy for reading trailers.
Debug with hex dumps and first-byte inspection. When a consumer rejects your payload, compare bin2hex($packed) against known fixtures. The first byte tells you the MessagePack type. Cross-check with msgpack-cli, another language's library, or the official spec. Most integration bugs are type mismatches (str vs bin, array vs map, int64 vs PHP int), not library bugs.
Options are validated at construction. Mutually exclusive combinations such as FORCE_STR | FORCE_BIN or FORCE_FLOAT32 | FORCE_FLOAT64 throw InvalidOptionException. Resolve the packer during application boot so configuration errors fail fast rather than on the first request.
Catch the right exceptions. PackingFailedException means the value cannot be represented. UnpackingFailedException means malformed input. InsufficientDataException means the buffer is incomplete, which is normal in streaming code and should be handled with tryUnpack() rather than treated as corruption.
Compression is a separate concern. MessagePack reduces structural overhead, but it does not compress repetitive strings. For large payloads, gzip or zstd the packed bytes and record that in your envelope. Do not compress tiny payloads; the framing overhead will outweigh the gain.
How can I help you explore Laravel packages today?