Msgpack Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require rybakit/msgpack
   

   Add to composer.json if using Laravel's autoloader:

   "autoload": {
       "psr-4": {
           "App\\": "app/",
           "MessagePack\\": "vendor/rybakit/msgpack/src/"
       }
   }
   

   Run composer dump-autoload.

2. First Use Case: Serialize/deserialize data in a Laravel job or API response:

   use MessagePack\Packer;
   use MessagePack\BufferUnpacker;
   
   // Pack (serialize)
   $packer = new Packer();
   $packed = $packer->pack(['key' => 'value', 'data' => [1, 2, 3]]);
   
   // Unpack (deserialize)
   $unpacker = new BufferUnpacker($packed);
   $unpacked = $unpacker->unpack();
   

Where to Look First

• Packing: Packer class and its methods (e.g., pack(), packMap()).
• Unpacking: BufferUnpacker class and its methods (e.g., unpack(), tryUnpack()).
• Custom Types: CanBePacked, CanPack, and Extension interfaces in src/Type* and src/TypeTransformer.
• Examples: /examples/MessagePack for real-world use cases (e.g., DateTimeExtension.php).

Implementation Patterns

Core Workflows

1. API Responses or Caching

Use MessagePack for compact binary responses (e.g., in Laravel's Response):

return response(MessagePack::pack($data), 200, [
    'Content-Type' => 'application/msgpack',
]);

Client-side handling:

fetch('/api/data', { headers: { Accept: 'application/msgpack' } })
    .then(res => res.arrayBuffer())
    .then(buffer => MessagePack.decode(new Uint8Array(buffer)));

2. Database or Cache Storage

Store serialized data in Redis or a binary column:

// Store
$packed = MessagePack::pack($model->toArray());
Redis::set('key', $packed);

// Retrieve
$data = MessagePack::unpack(Redis::get('key'));

3. Streaming Data

Handle chunked data (e.g., from a socket or file stream):

$unpacker = new BufferUnpacker();
while ($chunk = fread($stream, 1024)) {
    $unpacker->append($chunk);
    if ($messages = $unpacker->tryUnpack()) {
        foreach ($messages as $message) {
            // Process $message
        }
    }
}

4. Custom Type Handling

Extend for domain-specific types (e.g., Carbon instances):

use MessagePack\Extension;
use Carbon\Carbon;

class CarbonExtension implements Extension {
    public function getTypeId() { return 1; }
    public function pack($value, Packer $packer) { /* ... */ }
    public function unpack($value, Unpacker $unpacker) { /* ... */ }
}

$packer = new Packer();
$packer->extendWith(new CarbonExtension());
$packed = $packer->pack(Carbon::now());

Integration Tips

• Laravel Service Provider: Bind Packer/Unpacker as singletons for reuse:

  $this->app->singleton(Packer::class, function () {
      return new Packer(PackOptions::FORCE_MAP);
  });
  

• Middleware for MsgPack: Parse MsgPack requests/responses:

  public function handle($request, Closure $next) {
      if ($request->isJson() && $request->header('Content-Type') === 'application/msgpack') {
          $data = MessagePack::unpack($request->getContent());
          $request->merge(['msgpack_data' => $data]);
      }
      return $next($request);
  }
  

• Queue Jobs: Serialize payloads to reduce size:

  $job = (new ProcessDataJob($data))->onQueue('msgpack');
  $job->handle() {
      $unpacked = MessagePack::unpack($this->data);
      // Process $unpacked
  }
  

• Testing: Use MessagePack::pack()/unpack() in PHPUnit assertions:

  $this->assertEquals(
      MessagePack::unpack(MessagePack::pack(['key' => 'value'])),
      ['key' => 'value']
  );
  

Gotchas and Tips

Pitfalls

1. Integer Overflow: MsgPack supports 64-bit unsigned integers, but PHP may overflow. Use UnpackOptions::BIGINT_AS_STR or BIGINT_AS_GMP:

   $unpacker = new BufferUnpacker($packedData, UnpackOptions::BIGINT_AS_GMP);
   

2. String vs. Binary:

   • FORCE_STR/FORCE_BIN are mutually exclusive. Avoid mixing them.
   • Use DETECT_STR_BIN for automatic detection (slower but flexible).

3. Array/Map Detection:

   • DETECT_ARR_MAP may misclassify sparse arrays. Force with FORCE_ARR/FORCE_MAP if needed.

4. Streaming Edge Cases:

   • tryUnpack() may return partial messages if the stream ends mid-message. Validate data integrity.

5. Custom Extensions:

   • Extension IDs must be unique (0–127). Predefined IDs (e.g., 0 for timestamps) are reserved.

6. Floating-Point Precision:

   • FORCE_FLOAT32 loses precision. Use FORCE_FLOAT64 (default) for accuracy.

7. Circular References:

   • The library does not handle circular references in objects. Use json_encode() + json_decode() as a fallback.

Debugging Tips

• Inspect Packed Data: Use bin2hex() to debug binary output:

  $packed = MessagePack::pack(['test' => 123]);
  echo bin2hex($packed); // Output: 82a574657374a2616263
  

• Validate MsgPack: Use the msgpack.org validator or:

  composer require --dev msgpack/msgpack-php-ext
  php -r '$packed = file_get_contents("data.msgpack"); var_dump(msgpack_unpack($packed));'
  

• Performance Profiling: Compare PackOptions combinations:

  $packer = new Packer(PackOptions::FORCE_MAP | PackOptions::FORCE_STR);
  $time = microtime(true);
  $packed = $packer->pack($largeArray);
  echo microtime(true) - $time; // Measure overhead
  

Extension Points

1. Custom Transformers: Extend CanPack for non-object types (e.g., resources):

   class ResourceTransformer implements CanPack {
       public function canPack($value) { return is_resource($value); }
       public function pack($value, Packer $packer) { /* ... */ }
   }
   

2. Type Objects: Implement CanBePacked for domain objects:

   class User implements CanBePacked {
       public function pack(Packer $packer) {
           return $packer->packMap([
               'id' => $this->id,
               'name' => $this->name,
           ]);
       }
   }
   

3. Unpacker Hooks: Override BufferUnpacker to add pre/post-unpack logic:

   class CustomUnpacker extends BufferUnpacker {
       protected function unpackValue() {
           $value = parent::unpackValue();
           return $this->transform($value); // Custom logic
       }
   }
   

4. Batch Processing: Use BufferUnpacker for bulk unpacking:

   $unpacker = new BufferUnpacker();
   foreach ($batch as $chunk) {
       $unpacker->append($chunk);
       while ($unpacker->hasRemaining()) {
           yield $unpacker->unpack();
       }
   }
   

Config Quirks

• Default Options:

  • Packer: DETECT_ARR_MAP + FORCE_FLOAT64.
  • BufferUnpacker: BIGINT_AS_STR.

• Thread Safety: Packer/Unpacker instances are not thread-safe. Create new instances per request/