- What is cerbero/lazy-json-pages used for in a Laravel application?
- It is a read-side adapter that turns any paginated JSON API into an Illuminate\Support\LazyCollection, fetching pages through concurrent HTTP requests. Laravel teams typically drop it into their data-ingestion layer where they currently write while ($hasMore) { $page = $client->get(...); } loops, for example when pulling large CRM exports, marketplace catalogs, SaaS list endpoints, or accounting records into sync jobs. It is not a domain component, queue, or persistence layer, so keep it behind an adapter boundary instead of leaking raw arrays into business logic.
- How do I install cerbero/lazy-json-pages?
- Install it with Composer using composer require cerbero/lazy-json-pages. There is nothing to publish and no service provider to register, since the package is a genuinely small, thin piece of logic. Wire it into a service class or a queued job rather than sprinkling LazyCollection construction across the codebase.
- Which Laravel and PHP versions does lazy-json-pages support?
- It targets Laravel 9 through 11 with PHP 8.1 or newer, and Guzzle 7 sits underneath. Note that the "framework-agnostic" label is only half true: the package depends on illuminate/support and illuminate/http even outside a full Laravel app. Pin the major version and verify the Composer constraints against your own PHP and Laravel upgrade matrix before adopting it.
- How do I set up pagination for a JSON API with lazy-json-pages?
- You provide a request factory or URL builder plus a page-parsing callable that returns the page items together with metadata telling the package whether a next page exists. The LazyCollection then resolves pages lazily as you iterate, so you describe how pagination works once and consume results like any other Laravel collection. Keep that parsing callable small and isolated, because it is where provider-specific quirks live.
- Does lazy-json-pages keep memory usage constant for large APIs?
- No. Laziness means you never hold every page in memory at once, but the concurrent pool still keeps several pages in flight, so peak memory is roughly concurrency multiplied by page payload size. Teams often assume that lazy equals O(1) memory and are surprised when a large page size or high concurrency blows up a worker. Tune both values deliberately for your dataset.
- How does lazy-json-pages handle concurrent requests, and can I tune the concurrency?
- Pages are fetched with Guzzle promises in standalone mode and through Http::pool inside Laravel. Concurrency is what gives the package its throughput, but it also drives peak memory and how hard you hit the provider. Pick one transport path and standardize on it, since the two differ around exception propagation and connection reuse.
- How do I handle rate limits, retries, and failed requests with lazy-json-pages?
- The package has no first-class retry, backoff, or resume logic; whatever resilience you get is inherited from Guzzle or Laravel's HTTP client. You are responsible for jittered backoff on 429 responses, retry idempotency, and staying inside the provider's documented rate limits and concurrency tolerance. Add those policies in your adapter so they are testable and applied consistently.
- How should I test an adapter that uses lazy-json-pages?
- Keep the page-parsing callable isolated and unit test it against recorded fixture responses for each pagination shape, including empty and last pages. For integration confidence, run the new adapter in shadow mode beside your existing loop on a low-risk endpoint and diff item counts and per-item checksums. That catches pagination drift before you cut over a real integration.
- How can I be sure I fetched every page and did not silently truncate results?
- Silent truncation is the main risk: if the has-next-page predicate depends on a field the API changes or returns differently on empty results, you get a short collection with no error. Add a max-page guard and a reconciliation check that compares ingested counts or checksums against a known total. Also remember that offset pagination over mutable datasets produces duplicates and skips, so use dedupe keys or prefer cursor-based endpoints.
- What are the alternatives to lazy-json-pages, and is it still maintained?
- The closest alternative is a hand-rolled loop or Guzzle/Http::pool pool you control yourself; full ETL or scraper frameworks are heavier than most Laravel ingestion jobs need. The package is small, has roughly 195 stars, a single maintainer, and no commercial backing, which means it is effectively unvalidated at scale and you may end up owning it. Treat it as a small piece of code you can vendor or fork if upstream goes quiet.