Infatica Scraper Laravel Package

Getting Started

The package is a thin Laravel wrapper over the Infatica Scraper API, so the mental model is: install → configure credentials → inject the client → receive a DTO. Start by pulling it in and publishing whatever config it ships with:

composer require pisotskyi-el/infatica-scraper
php artisan vendor:publish --tag=infatica-config

Because the package lives on GitLab rather than Packagist, confirm the repository is declared in composer.json (a vcs entry pointing at https://gitlab.com/pisotskyi.el/infatica-scraper) and that your CI has access. If Composer can't resolve it, that's almost always the missing repository entry or a GitLab token, not a code issue.

Set your credentials in .env and never commit them:

INFATICA_API_KEY=your-key-here

Look first at the published config/infatica.php — it will tell you which env keys are actually read, the base URI, and any default timeout/retry values. Read the src/ tree next: the service provider tells you the container binding name, the Dto/ directory tells you the shape of a response, and the client class tells you which methods are public. That's usually a five-minute read for a package this size.

The first use case to prove out end-to-end is a single synchronous scrape: resolve the client from the container, pass it a URL, and dump the returned DTO. Once you can see a populated DTO, everything else (queues, batching, caching) is standard Laravel plumbing.

use PisotskyiEl\InfaticaScraper\Facades\Infatica; // adjust to actual namespace/facade alias

$result = Infatica::scrape('https://example.com/product/123');

dump($result->toArray());

If there's no facade, resolve the client interface from the container instead — the service provider binding is the source of truth.

Implementation Patterns

Wrap the client, don't sprinkle it. Create a small application-level service (App\Services\ScraperService) that depends on the Infatica client and owns retry policy, logging, and DTO→model mapping. This keeps the third-party namespace out of your controllers and makes it trivial to stub in tests or swap providers later.

Always scrape inside a queued job. Scraping is network-bound and slow. A job per URL (or per small batch) gives you Laravel's retry, backoff, and failed-job handling for free, and keeps HTTP response times clean.

class ScrapeProduct implements ShouldQueue
{
    use Queueable, InteractsWithQueue;

    public int $tries = 3;
    public array $backoff = [10, 60, 180];
    public int $timeout = 90;

    public function __construct(public string $url) {}

    public function handle(ScraperService $scraper): void
    {
        $dto = $scraper->scrape($this->url);
        Product::updateOrCreate(
            ['url' => $this->url],
            $dto->toArray()
        );
    }
}

Lean on the DTOs, not raw arrays. The package's value proposition is typed DTO support — use it. Type-hint the DTO in your own methods, persist via toArray(), and let the package handle hydration. If you find yourself reaching into a ->raw() or ->data array, check whether a DTO property already exists; if it genuinely doesn't, that's your signal to file an issue or write a small mapper rather than scattering data_get() calls.

Batch with concurrency limits. Scraping APIs typically charge per request and often throttle concurrency. Use Bus::batch() with a small ->allowFailures() set and dispatch in chunks rather than throwing hundreds of jobs at the queue at once:

Bus::batch(
    collect($urls)->map(fn ($url) => new ScrapeProduct($url))
)->allowFailures()->dispatch();

Cache aggressively for repeat scrapes. Product pages, listings, and metadata change slowly. A short-TTL cache keyed on the URL (or the scrape parameters) cuts cost dramatically:

$dto = Cache::remember("scrape:{$url}", now()->addHours(6), fn () => $scraper->scrape($url));

Model the failure modes explicitly. Wrap calls in a try/catch for the package's exception types (usually one base exception plus a transport/HTTP variant) and convert them into domain outcomes — a ScrapeFailed event, a status column on the record, or a dead-letter job. Don't let a 429 or 503 bubble up as a generic 500.

Test with Http::fake(). Since the wrapper ultimately speaks HTTP, faking the underlying request is the cleanest test seam. Assert on the request your service builds and feed back a canned response body to exercise DTO hydration without hitting the network.

Gotchas and Tips

• Credits, not just rate limits. Scraping APIs meter by request/credit. A runaway job or a retry storm can burn real money. Cap $tries, use exponential backoff, and never retry on 4xx client errors — only on timeouts and 5xx.
• Timeouts in two places. The config timeout and your queue job's $timeout are different guards. Set the job timeout higher than the HTTP timeout, or your job gets killed mid-request and the scraper may have already charged you.
• DTOs may be immutable. If properties are readonly, don't try to mutate them after hydration — build a new instance or map to your own model. Check the DTO constructor before assuming setters exist.
• Missing/null fields are normal. Scraped pages are inconsistent. Treat DTO fields as nullable unless the package documents otherwise, and validate before persisting to avoid NOT NULL column failures deep in a job.
• Facade aliasing. If the facade isn't auto-registered, add it to config/app.php or use the full class name. Facade resolution failures usually surface as "target class does not exist" and are a registration issue, not a bug.
• Version pin from GitLab. Since this isn't on Packagist, pin a specific tag/commit in composer.json rather than dev-main, or a force-push on the remote will silently change your dependency. Tag releases you depend on.
• Extension points are usually config + container binding. Before forking, check whether the config file exposes the HTTP client or base URI — overriding the binding in a service provider is often enough to inject a custom Guzzle handler (useful for logging or a test transport).
• Log request IDs. Scraping vendors return a request/job ID per call. Log it alongside the URL — when you're debugging a failed scrape with support, that ID is the only thing they can look up.
• Zero dependents means you're the QA. With no other users, edge cases aren't battle-tested. Read the wrapper's source (it's small) rather than trusting the README, especially around error handling and async/polling behavior if the API supports it.
• Watch for async endpoints. If Infatica offers asynchronous scraping, the "result" call may return a pending DTO. Don't treat a pending status as a success — assert on the status field before consuming the payload.