Weave Code
Search Text Transformer Laravel Package
becklyn/search-text-transformer
Converts HTML into clean, searchable plain text for indexing with search engines like Elasticsearch. Use SearchTextTransformer to strip tags and normalize content, making page text suitable for full‑text search and highlighting.
A library that extracts plain text from HTML for usage in search engines (like Elasticsearch)
Frequently asked questions about Search Text Transformer
- How do I integrate SearchTextTransformer with Laravel Scout for Elasticsearch?
- Override your model’s `toSearchableArray()` method to include the transformed text. Use the transformer in a service or facade to process HTML before passing it to Scout’s indexer. For example, add `SearchText::transform($model->content)` to your searchable attributes array.
- Can this package handle large-scale batch processing for nightly re-indexing?
- Yes, leverage Laravel’s queue system to process batches asynchronously. Wrap the transformer in a job (e.g., `TransformSearchTextJob`) and dispatch it for each model or batch. Monitor performance with large payloads (e.g., 1MB+ HTML) and cache results if reprocessing is frequent.
- What Laravel versions and PHP versions does SearchTextTransformer support?
- The package is tested for Laravel 8.x and PHP 8.x. Check the repository for the latest compatibility notes, but it should work seamlessly with Laravel 9/10 if no breaking changes are introduced. Always verify with your specific PHP version (8.0+ recommended).
- How can I customize the transformation to preserve headings (e.g., `<h1>`–`<h3>`) or ignore boilerplate?
- Extend the `SearchTextTransformer` class or wrap it in a service interface to override default rules. Use composition to inject custom logic (e.g., preserve headings by modifying the `stripTags` method or adding a pre-processing step). Fork the repo if heavy customization is needed.
- Does this package work with Algolia or other search backends besides Elasticsearch?
- Absolutely. Use the transformer in pre-indexing scripts or Laravel middleware to clean HTML before sending data to Algolia. For Scout, override `toSearchableArray()` as you would for Elasticsearch. The package is backend-agnostic—focus on transforming text for any search engine.
- How do I handle malformed HTML or edge cases like nested tables?
- Supplement the transformer with PHP’s `DOMDocument` for complex cases. Log or skip problematic HTML if needed, or implement fallback logic (e.g., return raw text if parsing fails). Test with your specific HTML structure to ensure robustness.
- Can I use this package in a headless CMS like Strapi or Craft CMS?
- Yes, integrate it into your CMS’s content pipeline. For example, use it in a Strapi custom field or Craft plugin to transform HTML before indexing. The transformer works anywhere PHP runs, making it ideal for decoupled architectures.
- What’s the best way to test this package in a Laravel application?
- Mock the transformer interface in unit tests (e.g., `HtmlToTextTransformer`) and use Laravel’s testing helpers to assert transformed output. Extend the package’s test suite with fixtures for your domain (e.g., product pages) and validate search relevance via user feedback or query logs post-integration.
- Should I store transformed text in the database or process it on-the-fly?
- Store it persistently if search performance is critical or if you need to reuse transformed text (e.g., for highlighting). Process on-the-fly for real-time use cases (e.g., API responses) or when storage isn’t feasible. Balance latency and consistency based on your workflow.
- Are there alternatives to SearchTextTransformer for Laravel?
- Consider `spatie/html-to-text` for simpler use cases or `symfony/dom-crawler` for advanced parsing. However, SearchTextTransformer is optimized for Laravel’s ecosystem (e.g., Scout, queues) and offers built-in extensibility. Evaluate based on customization needs and integration effort.
Weaver
How can I help you explore Laravel packages today?