Site Discovery Laravel Package

Getting Started

This package is part of the Capell CMS family (capell-app/site-discovery), so the first question before installation is whether you're adopting it standalone or alongside Capell. Check composer.json for hard dependencies on capell-app/* packages — if it pulls in the CMS core, it isn't a drop-in for a vanilla Laravel app.

Minimal path:

1. composer require capell-app/site-discovery
2. php artisan vendor:publish --provider="Capell\SiteDiscovery\SiteDiscoveryServiceProvider" (confirm the exact provider FQCN with php artisan vendor:publish and picking from the list — don't guess the tag)
3. Inspect config/site-discovery.php immediately after publishing. This is the file that determines 90% of day-to-day behaviour: which routes get registered, which models/pages feed the sitemap, and what gets emitted into robots.txt.
4. php artisan route:list --path=sitemap and --path=robots to confirm what the service provider actually registered. Some versions gate registration behind a config flag or an environment check, so an empty result is a config problem, not a bug.
5. Hit /robots.txt, /sitemap.xml, and /security.txt (whichever apply) in the browser.

Where to look first: the published config file, then src/Http/Controllers and src/Support in vendor/capell-app/site-discovery. The package is small enough that reading the source beats reading docs — the release history here suggests the README may lag the code.

First real use case: wire a sitemap entry source. Most adoption friction is here — the package needs to know which URLs are public. That's usually a config array of models, a contract/interface your models implement, or a callback. Find that extension point before writing any custom controllers.

Implementation Patterns

Config-as-source-of-truth. Resist the urge to hand-write a public/robots.txt or a sitemap controller. Keep everything in config/site-discovery.php, commit it, and let the package own the output. The moment you have both a static file and a dynamic route, you get silent divergence — the web server serves the static file and your route never runs.

Sitemap sources via model integration. The durable pattern is a small interface or trait on the models that represent public pages (posts, products, CMS pages), exposing getUrl(), getLastModified(), and a scope for "published only". Implement it once on a base model or a trait, then register the model classes in config. This keeps the sitemap correct as content changes without touching package code.

Cache the generated output. Sitemap generation is a full table scan by nature. Wrap the response in Cache::remember() keyed by locale/site, with a TTL measured in hours, and bust it from model observers (saved, deleted) on the classes feeding the sitemap. Alternatively generate to disk on a schedule (php artisan schedule:run with a nightly command) and serve the static artifact — better for large sites and for CDN caching.

Multi-locale and multi-domain. If you run localised routes (/en/..., /de/...) or multiple domains off one codebase, generate one sitemap per locale/domain and a sitemap index pointing at them. Check whether the package already does this or whether you need to loop over locales yourself — confirm from source rather than assuming.

Environment gating. Register the routes only in production (or explicitly opt in on staging). A publicly reachable sitemap on a staging box is a real indexing hazard, especially if staging isn't behind auth.

Testing. Feature-test the endpoints rather than unit-testing internals: assert 200, correct Content-Type (application/xml for sitemaps, text/plain for robots), that a known published model appears, and that an unpublished/draft model does not. The negative assertion is the one that catches real bugs.

Gotchas and Tips

Version discipline is the headline issue with this release. v1.0.49 again ships with nothing but a compare link, continuing the pattern from v1.0.48 — that's v1.0.43 → v1.0.49 with six patch releases and zero described changes. Semver says patch releases are safe, but a compare URL is not release notes, and a changelog that never gets written is a changelog you can't rely on. Before upgrading: pin your current version, run composer update capell-app/site-discovery --dry-run to see what moves, then read the actual diff (https://github.com/capell-app/site-discovery/compare/v1.0.43...v1.0.49) rather than trusting the changelog. Treat each upgrade as low-risk-but-unverified and cover it with the endpoint tests above.

Static public/robots.txt wins. Nginx/Apache will serve a physical file before Laravel ever routes. Delete it before relying on the dynamic route, and check public/ during debugging.

Config cache. php artisan config:cache after every config edit, or you'll debug a change that never took effect. Same for route:cache when route registration is conditional.

Sitemap protocol limits. 50,000 URLs and 50 MB uncompressed per file, and every URL in a sitemap must be same-host as the sitemap itself. If you're near either limit, split into multiple files behind a sitemap index. Cross-host URLs silently fail validation in Search Console.

Trailing slashes and canonical mismatches. The most common real-world failure isn't a crash — it's a sitemap full of URLs that 301-redirect or don't match your <link rel="canonical">. Assert that generated URLs equal the canonical URLs your app emits; a redirecting sitemap wastes crawl budget and produces "Page with redirect" warnings.

Ordering and lastmod honesty. Only emit <lastmod> when the timestamp reflects genuine content change. updated_at bumped by a counter or a re-save will train crawlers to distrust your lastmod entirely.

Extension points to prefer. If the package exposes events (e.g. before-render on the sitemap) or an interface for URL providers, use those rather than subclassing controllers in app/. Controller subclassing breaks on patch releases — which, given the changelog quality here, you will be taking somewhat blind.

Debugging checklist when output looks wrong: route:list (is the route registered?), config('site-discovery') via tinker (did config load?), the model scope (are rows actually published?), then the response headers. Only after those four does it make sense to open the package source.

Exit ramp. Keep the surface small — config, one trait, one caching wrapper — so that if the package stalls or a v1.x release changes behaviour undocumented, you can replace it with ~100 lines of your own controller. Given the sparse release notes, avoid coupling your domain models deeply to package-specific base classes.