Weave Code
Json Streamer Laravel Package
symfony/json-streamer
Symfony JsonStreamer provides powerful methods to read and write data structures from and into JSON streams, enabling memory-efficient processing of large JSON documents without loading everything at once.
Provides powerful methods to read/write data structures from/into JSON streams.
Frequently asked questions about Json Streamer
- How do I install symfony/json-streamer in a Laravel application?
- Install it with Composer using composer require symfony/json-streamer, picking the release line that matches your PHP version and Symfony dependencies. There is no Laravel service provider to register, so you simply resolve or inject the reader and writer classes where you need them. In practice you pair it with symfony/serializer, which Laravel already ships, to convert objects to arrays during streaming. After installing, run composer dump-autoload and rebuild any compiled container with php artisan optimize:clear followed by php artisan optimize.
- Is symfony/json-streamer compatible with Laravel 10 and Laravel 11?
- Yes, indirectly, because the component depends on symfony/serializer rather than on Laravel internals. Laravel 10 and 11 bundle Serializer components that satisfy the requirement, so the JsonStreamer 7.x line works on the PHP 8.2+ runtimes those versions target. The Symfony 8.x line of the package requires PHP 8.4, so only adopt it once your app and CI run that version. For Laravel 9 projects, stay on a Symfony 6.x-compatible release.
- How do I stream a large JSON export in Laravel without running out of memory?
- Use the JsonStreamer writer to emit records one at a time into a PHP stream instead of building a giant array and calling json_encode. For HTTP endpoints, open php://output or a temporary stream and return a StreamedResponse whose callback writes batches as they are queried. Because the API is generator-based, memory stays roughly constant no matter how many rows you export. Disable response-buffering middleware and close the stream when the callback finishes.
- How can I read a multi-gigabyte JSON file item by item in a Laravel queue job or Artisan command?
- Open the file with fopen in read mode, pass the handle to the JsonStreamer reader, and iterate the returned generator with foreach. Each iteration yields one decoded item, so you can validate it, dispatch follow-up work, or insert rows in chunks without holding the whole document. This pattern suits queue workers and long-running Artisan commands where memory_limit would otherwise be exhausted. Wrap the loop in try and finally so the file handle is closed even when a job fails or is retried.
- What is the difference between JsonStreamer and json_decode or json_encode for large payloads?
- json_decode and json_encode build the entire PHP structure or JSON string in memory, so peak usage grows linearly with document size and easily exceeds memory_limit above tens of megabytes. JsonStreamer reads and writes incrementally, giving constant memory usage relative to the number of items processed. The trade-off is item-at-a-time handling, so you cannot run whole-document operations or random access on the data. For small payloads the built-in functions are simpler and perfectly adequate.
- Does JsonStreamer work with Eloquent models and the Symfony Serializer in Laravel?
- JsonStreamer is built on Symfony Serializer normalizers, so any object graph the serializer can handle can be streamed once it is normalized to arrays. In practice you convert each Eloquent model with toArray or a dedicated normalizer and hand the array to the streamer to write. Complex graphs with recursive relations, custom value transformers, or synthetic properties may need custom normalizers. Test self-referencing objects explicitly, since recursion is a known edge case for streaming serializers.
- How do I test JsonStreamer code in Laravel with PHPUnit or Pest?
- There are no Laravel-specific test helpers, so ordinary unit tests work: write to a php://temp or php://memory stream, rewind it, and assert the decoded output matches the expected array. For readers, build JSON fixtures as strings or temp files and assert the items your generator yields in order. In feature tests you can hit a streaming route, but note that some test clients buffer the body. Add one larger fixture guarded by a memory assertion to catch regressions in streaming behavior.
- Is symfony/json-streamer stable and safe for production?
- The component follows Symfony's release process, and the 8.1.x line has shipped multiple patch releases with no breaking changes or deprecations, which is a strong maturity signal. Version 8.1.9 also integrates with Symfony DI container compilation, so registration happens at build time and remains compatible with config:cache, artisan optimize, and Octane preloading. The main caveats are runtime handle management in long-lived processes and resource retention during container builds. Validate with a clean optimize:clear and optimize cycle plus an Octane reload before deploying.
- How do I avoid memory or file handle leaks with JsonStreamer under Laravel Octane and queue workers?
- Always close streams and unset generator references in a finally block, because long-lived workers keep resources alive between requests and jobs. Under Octane, monitor open file descriptors with lsof against the worker PID during soak tests and watch RSS for slow growth. During deployment, check container build time and peak memory on composer install and artisan optimize, since streamable resources are now retained across compiler passes until StreamablePass ordering completes. Explicit cleanup is far cheaper than debugging a leak in production.
- What are the alternatives to JsonStreamer for handling large JSON in Laravel?
- The closest alternative is halaxa/json-machine for pull-based parsing of huge files, plus manual chunked parsing with json_decode, which gets fragile for nested documents. For database-driven exports, Laravel's lazy and cursor methods combined with a streamed response solve the memory problem without any extra package. JsonStreamer is the best fit when you already use Symfony Serializer and need symmetric read and write streaming with object normalization. If you only need to read one enormous file once in a CLI script, json-machine may be simpler.
Weaver
How can I help you explore Laravel packages today?