Word Extract Laravel Package

Getting Started

Install the package via Composer:

composer require bitandblack/word-extract

First Use Case: Extract words with a minimum length from a string (e.g., user-generated content, blog posts, or search queries).

use BitAndBlack\WordExtract\WordExtractor;

// Initialize with minimum word length (e.g., 5 characters)
$extractor = new WordExtractor(5);

// Extract words from a string
$words = $extractor->getWords('Laravel is an amazing PHP framework');
/*
Output:
[
    'amazing', // 7 chars
    'framework' // 9 chars
]
*/

Where to Look First:

• WordExtractor class: Core functionality for extraction and word handling.
• Callback pattern: Use getWithWordsHandled() to transform words (e.g., wrap in HTML, log, or validate).
• README examples: Quick reference for basic and advanced usage.

Implementation Patterns

1. Basic Word Extraction

Use getWords() to filter words by minimum length in:

• Content Moderation: Flag long words (e.g., spam, profanity).
• SEO Tools: Extract keywords for metadata or analytics.

$extractor = new WordExtractor(4);
$keywords = $extractor->getWords('The quick brown fox jumps over the lazy dog');
/*
Output:
[
    'quick', 'brown', 'fox', 'jumps', 'lazy', 'dog'
]
*/

2. Word Transformation via Callbacks

Process extracted words dynamically (e.g., logging, sanitization, or enrichment):

$handler = fn(string $word) => "<strong>{$word}</strong>";
$highlighted = $extractor->getWithWordsHandled(
    'Laravel is powerful',
    $handler
);
/*
Output: "Laravel is <strong>powerful</strong>"
*/

3. Laravel Integration Patterns

Service Provider

Register as a singleton for app-wide access:

// app/Providers/AppServiceProvider.php
public function register(): void {
    $this->app->singleton(WordExtractor::class, fn() =>
        new WordExtractor(config('word-extractor.min_length'))
    );
}

Access via dependency injection:

public function __construct(private WordExtractor $extractor) {}

Facade (Optional)

Create a facade for cleaner syntax:

// app/Facades/WordExtractorFacade.php
public static function extract(string $text): array {
    return resolve(WordExtractor::class)->getWords($text);
}

Usage:

$keywords = WordExtractorFacade::extract('Extract keywords here');

Artisan Command

Batch-process files (e.g., logs, CSV exports):

// app/Console/Commands/ExtractKeywords.php
public function handle(): void {
    $files = Storage::files('path/to/texts');
    foreach ($files as $file) {
        $content = file_get_contents($file);
        $keywords = $this->extractor->getWords($content);
        // Save or log $keywords
    }
}

Middleware

Preprocess incoming requests (e.g., extract keywords from form data):

// app/Http/Middleware/ExtractKeywords.php
public function handle(Request $request, Closure $next) {
    $keywords = $this->extractor->getWords($request->input('content'));
    $request->merge(['keywords' => $keywords]);
    return $next($request);
}

Event Listeners

Trigger extraction on model events (e.g., created for blog posts):

// app/Listeners/ExtractPostKeywords.php
public function handle(Created $event) {
    $keywords = $this->extractor->getWords($event->post->content);
    $event->post->update(['keywords' => json_encode($keywords)]);
}

4. Configuration

Store min_length in config/word-extractor.php:

return [
    'min_length' => env('WORD_EXTRACT_MIN_LENGTH', 5),
];

Access via:

$extractor = new WordExtractor(config('word-extractor.min_length'));

5. Testing

Mock the extractor in Laravel tests:

// tests/Feature/KeywordExtractionTest.php
public function test_extracts_keywords() {
    $extractor = Mockery::mock(WordExtractor::class);
    $extractor->shouldReceive('getWords')
        ->with('Test content')
        ->andReturn(['Test', 'content']);

    $this->app->instance(WordExtractor::class, $extractor);
    // Test your logic...
}

Gotchas and Tips

Pitfalls

1. Unicode/Non-Latin Characters:

   • The package does not handle Unicode word boundaries (e.g., café, über).
   • Workaround: Preprocess text with mb_split() or use preg_split('/\p{L}+/u').

   $words = preg_split('/\p{L}+/u', $text, -1, PREG_SPLIT_NO_EMPTY);
   

2. Hyphenated Words:

   • Treats hyphens as word separators (e.g., state-of-the-art → state, of, the, art).
   • Workaround: Use a callback to rejoin hyphenated terms or adjust regex.

3. Performance with Large Texts:

   • Linear time complexity: O(n) where n = string length. Test with inputs >10K characters.
   • Workaround: For bulk processing, use Laravel Queues or chunk the text.

4. Dependency on bitandblack/helpers:

   • The package requires bitandblack/helpers (v2.0+), which may introduce hidden dependencies.
   • Workaround: Audit helpers for conflicts (e.g., global functions overriding Laravel’s).

5. Edge Cases in Word Splitting:

   • Apostrophes (e.g., don't → don, t) or contractions may split unintentionally.
   • Workaround: Use a callback to merge split contractions or preprocess with regex.

6. PHP 8.2+ Requirement:

   • Blocks use in Laravel <10.x or PHP <8.2 environments.
   • Workaround: Use a containerized PHP 8.2+ environment or fork the package for older PHP.

Debugging Tips

1. Log Extracted Words:

   $words = $extractor->getWords($text);
   Log::debug('Extracted words', ['words' => $words]);
   

2. Validate Input: Ensure input is a string to avoid TypeError:

   if (!is_string($text)) {
       throw new InvalidArgumentException('Input must be a string');
   }
   

3. Test with Edge Cases:

   $testCases = [
       'Unicode: café, über', // Fails without Unicode support
       'Hyphenated: state-of-the-art', // Splits hyphens
       'Contractions: don\'t, can\'t', // Splits apostrophes
       'Numbers: 12345', // Treats as word if ≥ min_length
   ];
   

4. Profile Performance: Use Laravel Debugbar or Xdebug to measure extraction time for large texts.

Extension Points

1. Custom Word Splitting: Override the default splitWords() method:

   class CustomWordExtractor extends WordExtractor {
       protected function splitWords(string $text): array {
           return preg_split('/\W+/', $text, -1, PREG_SPLIT_NO_EMPTY);
       }
   }
   

2. Add Stopword Filtering: Extend the class to exclude common words:

   class StopwordAwareExtractor extends WordExtractor {
       private array $stopwords = ['the', 'and', 'is'];
   
       public function getWords(string $text): array {
           $words = parent::getWords($text);
           return array_filter($words, fn($word) => !in_array(strtolower($word), $this->stopwords));
       }
   }
   

3. Integrate with Laravel Cache: Cache extraction results for repeated queries:

   public function getWords(string $text): array {
       $cacheKey = 'word_extract:' . md5($text);
       return Cache::remember($cacheKey, now()->addHours(1), function() use ($text) {
           return parent::getWords($text);
       });
   }
   

4. Add Language Support: Use mb_split() for multilingual text:

   protected function splitWords(string $text): array {
       return mb_split('/\p{L}+/u', $text, -1, MB_SPLIT_NO_EMPTY);
   }
   

Configuration Quirks

1. Default min_length: The package has no default; always specify a value in the constructor or config.

2. Case Sensitivity: Extraction is case-sensitive (e.g., Laravel and laravel are treated as different words). Workaround: Normalize case before extraction:

   $ext