Weave Code
Weaver
Helps Laravel developers discover, compare, and choose open-source packages. See popularity, security, maintainers, and scores at a glance to make better decisions.
Feedback
Share your thoughts, report bugs, or suggest improvements.
Subject
Message
Robots Txt Bundle

Robots Txt Bundle Laravel Package

miisieq/robots-txt-bundle

A Symfony bundle for generating and serving a dynamic robots.txt file, letting you define user-agent rules, disallow paths, and sitemap entries in configuration instead of a static file. Handy for multi-environment setups where crawl rules need to vary.

View on GitHub
Deep Wiki
Context7

miisieq/robots-txt-bundle is a Symfony bundle for serving a dynamic, configuration-driven robots.txt. Define crawler rules, sitemap locations, and host directives in configuration instead of maintaining a static file, keeping behavior consistent and easy to adjust.

  • Configurable Allow/Disallow rules and user agents
  • Environment-aware robots.txt generation
  • Sitemap and host directive support
  • Symfony routing/controller integration
  • No manual static file maintenance
Frequently asked questions about Robots Txt Bundle
Can I install miisieq/robots-txt-bundle in a Laravel application?
Not cleanly. It is a Symfony bundle that relies on Symfony's bundle registration, DI extension, and config-tree conventions, none of which Laravel provides. Composer will also likely fail on the platform check, because the bundle targets the PHP 5.5 to 7.x era while Laravel 10 and 11 require PHP 8.1 or newer. Forcing it with ignore-platform-reqs and hand-wiring a Symfony container inside Laravel is an anti-pattern for production.
What does miisieq/robots-txt-bundle actually do?
It generates and serves a dynamic robots.txt file from configuration instead of a hand-maintained static file. Based on the Symfony environment, production allows the whole site to be indexed while any other environment blocks indexing entirely. You can configure user agents, allow and disallow rules, and sitemap directives in YAML or PHP config.
Is miisieq/robots-txt-bundle still maintained?
No. The last release was 2018-10-15, the repository is archived, and it has roughly three stars and zero dependents. There is no maintainer, no security response path, and no PHP 8 compatibility work. For a dependency you cannot patch, that lifecycle risk outweighs the fact that robots.txt generation is a small, low-complexity problem.
Which PHP and Laravel versions does miisieq/robots-txt-bundle support?
Its constraints reflect the 2018 Symfony 2, 3, and 4 era, which means PHP 5.5 through 7.x and Symfony kernel conventions. It has no declared support for Laravel at all, and no release covers PHP 8. Modern Laravel versions are therefore a hard compatibility blocker rather than a configuration issue.
How do I generate a dynamic robots.txt in Laravel without this package?
Add a config file with user_agent, allow, disallow, and sitemap keys, then register a single route that returns a text/plain response built from those values. The generator itself is a short string builder, so a small support class or controller action covers the 80 percent case in well under a day including tests. You keep full control, no foreign abstraction layer, and no abandoned dependency.
Is a static public/robots.txt enough, or do I need dynamic generation?
For most Laravel applications a versioned static public/robots.txt managed through deploy configuration is sufficient and adds zero code. Dynamic generation only earns its keep when rules genuinely change per environment, per domain, or per tenant, or when sitemap URLs must be resolved at runtime. Decide which of those drivers you actually have before adding any code.
How do I block staging and non-production environments from being indexed in Laravel?
Check the app environment inside your robots.txt route and return a full disallow block when it is not production. As a belt-and-braces measure, also send an X-Robots-Tag noindex header or protect the environment with HTTP basic auth. This is a few lines of native Laravel and does not require a Symfony bundle or any third-party package.
Does miisieq/robots-txt-bundle support sitemap directives and per-user-agent rules?
The documented configuration covers user agents, allow and disallow rules, and sitemap directives from YAML or PHP config, so per-agent rule grouping is part of the model. Crawl-delay and other less common directives are not described in the overview, so verify against the source if you depend on them. Either way, this is just text emission that a short in-house builder handles.
How do I test a dynamic robots.txt route in Laravel?
Write an HTTP test that asserts a 200 status, a text/plain content type, and the presence of the expected Disallow and Sitemap lines in the body. Add a second test that fakes the non-production environment and asserts the block-everything output. Because the response is plain text, assertions on the raw body are stable and fast.
What should Laravel teams use instead of miisieq/robots-txt-bundle?
Build it natively: a config-driven route returning text/plain, a static file where rules are stable, or a maintained Laravel-native package with recent releases and PHP 8 support if you insist on a dependency. Treat this bundle as a reference implementation for the idea, not as installable code. Do not adopt an archived Symfony bundle into a Laravel codebase.
Weaver

How can I help you explore Laravel packages today?

Conversation history is not saved when not logged in.
Prompt
Add packages to context
No packages found.
elsayed85/laravel-api-exceptions
ouhssini/laravel-demo-mode
daworks/ncloud-outbound-mailer-driver
gp247/cms
ignibyte/aic-gates
nexa-merchant/blog
novasa/laravel-language-center
opencage/geocode
adolfoholzer/parameters
livenetworks/ln-starter
shitoudev/permissionmanager
johannesclimacus/laravel-tui
hocvt/seotools
vitalybaev/sypexgeo
amphibee/themosis-ignition
gito-me/fast-excel
frankyso/laravel-ipaymu
geolize-studio/laravel-chile
boomerangui/chat
cgv/localized-resource