Querylist Laravel Package

Technical Evaluation

Architecture fit. QueryList is a framework-agnostic scraping library built around a fluent, jQuery-style API (QueryList::get($url)->find('a')->attrs('href')). It exposes three integration surfaces: a singleton (QueryList::getInstance()), a global helper (pq()), and static global config (QueryList::config()). All three are anti-patterns inside a Laravel container-managed app — they bypass config repositories, break test isolation, and leak state between jobs in long-running workers (Octane, queue daemons). The correct architectural posture is to treat QueryList as an internal implementation detail behind a PageExtractor contract: one adapter class owns the singleton, reads config from config/, returns typed DTOs, and is faked in tests. With that wrapper the fit is acceptable; without it, it will corrode the codebase.

Technical risk. Several risks are architectural rather than incidental:

• Dependency lineage. HTML parsing is delegated to jaeger/phpquery-single, a maintained fork of phpQuery (a ~2009 codebase). The CSS selector engine is jQuery-flavored, not W3C-compliant, and does not use symfony/css-selector. Complex selectors (:nth-child, attribute operators, namespaces) may behave differently than engineers expect from DomCrawler or browser DevTools — a class of bug that only surfaces on specific pages.
• Single-maintainer concentration. One primary author, bilingual issue tracker, and a release cadence that is hard to forecast. The "last release 2026-02-02" date should be verified as a substantive release versus a dependency/CI bump before it is read as a health signal.
• Static global state. QueryList::config() and the singleton make per-tenant proxy, timeout, or user-agent configuration awkward and unsafe under concurrency.
• No PSR-18/PSR-7 transport abstraction. HTTP is Guzzle/curlmulti-bound, so you cannot swap in Laravel's HTTP client to inherit its retry middleware, event hooks, or Http::fake() testing story. Concurrency is handled by curl_multi rather than promises, which does not compose with Laravel's queue/batch primitives.
• Silent failure mode. A stale selector returns an empty collection rather than throwing. Without explicit assertions this degrades into silent data loss in production.
• Legal/compliance surface. No robots.txt enforcement, no rate limiter, no domain allowlist. If target URLs are ever user- or config-supplied, this is an SSRF vector that must be gated at the wrapper boundary.

Key questions for the vendor/maintainer and the spike.

1. What is the supported PHP version matrix (8.2/8.3/8.4) and does it conflict with Laravel's Guzzle ^7 pin?
2. Is there a tagged release policy, a CHANGELOG, and a security disclosure channel?
3. What is the license (confirm MIT) and are there any transitive licenses in the phpQuery fork that need review?
4. Does the library expose per-instance configuration for timeout, proxy, headers, and concurrency, or is global config the only path?
5. What is the practical memory ceiling per document, and is there streaming/incremental parsing for large pages?
6. Which encoding handling is built in — does it auto-detect GBK/Shift-JIS, or is mb_convert_encoding on us?

Integration Approach

Stack fit. QueryList sits cleanly in the "extraction" tier of a Laravel application: Http-independent fetch → parse → normalize into DTOs → persist. It should never be called from a controller. The idiomatic shape is an artisan command or queued job that resolves an injected PageExtractor, which internally delegates to QueryList. Configuration (base URLs, selectors, timeouts, proxy pool, concurrency) belongs in config/scraping.php, not in QueryList::config() calls scattered through the codebase.

Compatibility. Composer-installable with no Laravel service provider and no auto-discovery, so registration is manual and explicit — which is fine. The real compatibility work is dependency resolution: confirm the package's Guzzle constraint against the application's existing guzzlehttp/guzzle version, and confirm PHP 8.x support in the specific minor you pin. Isolate this in a throwaway branch first; a Guzzle downgrade would ripple across every HTTP integration in the app.

Migration path. Two realistic origins:

• From Goutte / Symfony DomCrawler: mechanically similar ($crawler->filter('a')->each() → $ql->find('a')->texts()), with the caveat that selector semantics differ at the edges — budget a regression pass over every existing selector rather than assuming parity.
• From regex or DOMDocument scraping: strictly an upgrade in readability; migrate opportunistically when a scraper next breaks.
• From an existing headless-browser scraper (Puppeteer/PhantomJS): QueryList is not a substitute. It does not execute JavaScript. JS-rendered targets require keeping a browser tier and treating QueryList as the static-HTML fast path.

Sequencing.

1. Spike (1–2 days): install in a branch, resolve dependencies, scrape three representative real targets, measure memory and wall time, verify encoding handling on one non-UTF-8 page.
2. Wrapper: define PageExtractor interface plus a QueryList adapter; bind in a service provider; fake in tests. No QueryList type hints outside the adapter.
3. Vertical slice: one target end-to-end as a queued job — fetch, parse, validate, persist — with a schema/DTO contract and explicit "zero results" failure.
4. Observability: log selector hit counts, item counts, and parse duration per run. This is the only reliable defense against silent selector drift.
5. Harden: rate limiter, proxy rotation, retry/backoff, robots/domain allowlist — all implemented in Laravel, not in the library.
6. Broaden: port remaining targets one at a time, each with its own fixture-based test using saved HTML.

Operational Impact

Maintenance. The library itself is small and stable; the maintenance burden is the wrapper plus the selectors, not the dependency. Pin an exact version, never ^ on a single-maintainer package, and track upstream commits between releases. Have an explicit contingency: because the API surface you actually use is narrow (fetch, find, text/attr/html, rules), swapping to DomCrawler behind the same PageExtractor interface is a contained 1–2 day job — that optionality is the real reason to insist on the wrapper.

Support. Documentation and the largest body of community Q&A are Chinese-first. English-language support is essentially the GitHub issue tracker and reading the source. This is not a blocker for a competent PHP team but it does raise the cost of every unusual question, and it means an English-speaking on-call engineer debugging a 2 a.m. parse failure has a thinner safety net. Plan for one team member to become the internal owner and to write the runbook.

Scaling. QueryList scales within a process, not across a cluster. Its curl_multi concurrency is bounded by memory and file descriptors, and each job re-fetches its targets. Horizontal scale comes from queue workers (php artisan queue:work on N replicas), which means throughput is governed by your queue infrastructure and your politeness budget toward target sites — not by the library. Plan proxy egress and per-domain concurrency caps before, not after, the first ban. Expect CPU-bound parsing to be the bottleneck on large documents, and memory to spike on pages with very large or deeply nested DOMs; enforce a response-size ceiling at fetch time.

Failure modes to instrument for.

• Selector drift (highest frequency, lowest visibility): target markup changes, parse returns empty, pipeline reports success. Mitigate with hard assertions on expected item counts and alerting on sudden volume drops.
• Blocking (403/429/CAPTCHA): requires rate limiting, realistic headers, and proxy rotation — none provided by the library.
• Timeouts and partial pages: a truncated body yields partial extraction with no error.
• Encoding corruption: non-UTF-8 sources silently mangle text unless normalized.
• Memory exhaustion: oversized documents killing a worker mid-batch.
• Compliance exposure: scraping targets whose terms of service prohibit it; no technical control prevents this, so it needs a policy gate in code review.

Ramp-up. A developer fluent in jQuery or CSS selectors is productive in half a day and production-safe in about a week — the week being spent on the Laravel-side concerns QueryList does not solve (queues, retries, rate limits, observability, test fixtures), not on the library. The steeper learning curve for a TPM is organizational: establishing the selector-fixture test discipline and the wrapper boundary before the first scraper ships, because retrofitting either after five targets are live is materially more expensive than doing it up front.