- How do I install j0k3r/graby-site-config in a Laravel project?
- Run `composer require j0k3r/graby-site-config` in your project root. The package integrates seamlessly with Laravel’s HTTP client and requires no additional setup for basic usage. Ensure your Laravel version is 9 or higher and PHP is 8.1+.
- What types of site configurations does this package support?
- The package supports fetching and parsing XML (e.g., sitemaps, robots.txt), JSON, and plaintext configs. It’s designed for site-specific extraction rules like XPath selectors, filters, and cleanup logic to improve content parsing quality for scrapers or crawlers.
- Can I cache fetched configurations to improve performance?
- Yes, leverage Laravel’s cache system (e.g., Redis, file) to store fetched configs. Use `Cache::remember()` or dispatch async jobs with Laravel Queues to avoid repeated HTTP requests. The package doesn’t enforce caching but works natively with Laravel’s caching layer.
- Does j0k3r/graby-site-config work with Laravel Queues for async fetching?
- Absolutely. Dispatch config-fetching logic as a queued job (e.g., `FetchConfigJob`) to process requests asynchronously. This is ideal for large-scale scraping where performance and rate limits are concerns. Pair it with Laravel’s cache for further optimization.
- What Laravel versions and PHP requirements does this package support?
- The package is compatible with Laravel 9 and 10, and requires PHP 8.1 or higher. For older Laravel versions (e.g., 8.x), check the `composer.json` constraints or use a standalone PHP setup with Guzzle. Always test thoroughly in your environment.
- How do I handle rate limiting or throttling with this package?
- The package doesn’t include built-in rate limiting, but you can add middleware to Laravel’s HTTP client (e.g., Guzzle’s rate limiter) or use Laravel’s `throttle` middleware. For high-volume scraping, implement retries with exponential backoff in your job or service layer.
- Is this package secure for fetching untrusted remote configs?
- No, the package doesn’t validate remote configs by default. For untrusted sources, validate fetched data against schemas (e.g., using `spatie/fork`) or sandbox parsing logic. Avoid blindly trusting configs to prevent SSRF or injection risks, especially if configs include dynamic URLs.
- What’s the difference between using this package vs. Laravel’s built-in HTTP client?
- This package adds value for parsing, caching, and site-specific config logic (e.g., XPath, filters) tailored for scraping. If you only need raw HTTP requests, Laravel’s `Http::get()` is sufficient. Use graby when you need pre-configured rules for domains or async processing.
- How do I extend or customize the package for my use case?
- Extend the package by creating Laravel events (e.g., `ConfigFetched`) or middleware for the HTTP client. For example, add custom headers, auth, or parsing logic. The package is lightweight and designed to integrate with Laravel’s Service Container for dependency injection.
- What should I do if the package stops being maintained?
- Since this is a fork of FiveFilters’ repo, contributions must go upstream. If maintenance stalls, fork the repo yourself or refactor to use Laravel’s HTTP client directly. The package’s core logic (fetching/parsing) is simple enough to replicate with minimal effort.