- What does awcodes/botly actually do: manage robots.txt or detect bots?
- Botly is a robots.txt management package for Laravel: it stores crawler rules in a database table and serves a dynamic robots.txt, with a Filament admin page for editing those rules. Some package metadata and blurbs describe a bot-detection middleware framework with webhook and conversation handling, which does not match the repository tagline or the v1.0.6 release notes. Treat that description as stale or copied from another package and verify the actual artifact in the source before adopting it. If you need request-time bot detection or crawler logging, do not assume Botly provides it.
- Do I need Filament to use Botly?
- Filament is effectively the reason to choose Botly, because the admin page is the main value it adds over a committed static file. Without Filament you would be adopting a rules table and a string renderer and then building your own editing UI, at which point writing your own route and model is usually simpler. Confirm in the package source whether Filament is a hard Composer dependency or an optional integration before planning the install.
- How do I install Botly in a Laravel application?
- The install path is standard Laravel: require the package with Composer, publish its config file and migrations, run the migration to create the rules table, and register the Filament page in your panel provider. Then reproduce your current static robots.txt as rules and diff the rendered output against the file you serve today. A competent Laravel developer can validate this on a canary environment in roughly half a day.
- Which Laravel, PHP, and Filament versions does Botly support?
- The package is at v1.0.6, a patch release with no API boundary change, but the published documentation does not state a formal support matrix or deprecation policy. Check composer.json constraints and the release notes for the exact Laravel, PHP, and Filament majors before installing, and pin the tagged version you validate. Because there is a single maintainer and no LTS commitment, read diffs before bumping.
- Is robots.txt rendered on every request, or generated when rules are saved?
- The design described is a renderer that emits robots.txt, and the main technical concern is request-time PHP rendering on every crawler hit, which is a poor default if generation cannot be cached to disk. Confirm whether generation happens on request, on save, on a schedule, or through an artisan command. If it is request-time only, add a cache or write the output to disk so crawler traffic does not hit PHP and the database on every fetch.
- What happens if both a static robots.txt file and Botly's database rules exist?
- This is the most important integration question and it is not clearly documented: the web server may serve a committed robots.txt before Laravel ever reaches Botly's route. Decide explicitly whether the package overrides, merges with, or defers to a static file, and test it on a canary environment. Remove or redirect the static file if the database rules are meant to win, and keep a copy as a rollback artifact.
- What is served if the database is unavailable?
- If robots.txt is rendered at request time, a database outage can return a 500 on that path, which is a real SEO incident because crawlers may back off or drop directives. Plan for a fallback, such as a disk-cached copy or a static file the web server serves independently. Test the failure mode deliberately before you rely on the package in production.
- Can I write the generated robots.txt to disk from a deploy hook?
- The package is oriented around dynamic serving, and a disk-write or deploy-hook workflow is not evidenced in the documentation. Check whether an artisan command or service method exists that returns the rendered string so you can write it to public/robots.txt during deployment. If not, wrap the renderer yourself and cache the output, which also removes the per-request database hit.
- How do I test a robots.txt change before it hits production?
- Reproduce your current static file as rules in a staging environment, render the output, and diff it against the file you serve today. The v1.0.6 casing normalization for Disallow, Allow, Crawl-delay, and Clean-param can change output formatting, so update snapshot tests and any committed or CDN-cached copies. Treat production rule edits like a deploy: require a named approver and keep the previous static file as a rollback artifact, because this UI makes deindexing easy.
- Is Botly actively maintained, and what are the reasonable alternatives?
- Botly has a single maintainer, very few stars, no dependents, external documentation, and no stated deprecation policy or LTS, so pin the tagged version and review diffs before upgrading. It is fine as an SEO convenience when Laravel and Filament are already your stack, but it is not a contractual dependency. Alternatives include keeping a committed static robots.txt behind a pull request, or writing a small route, migration, and model yourself if you want database-driven rules without the Filament dependency.