Polyfill Iconv Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require symfony/polyfill-iconv
   

   • Add to require (not require-dev) if needed in production.
   • No additional configuration is required; the polyfill auto-loads when ext-iconv is missing.

2. First Use Case: Test basic encoding conversion in a Laravel controller or command:

   use Illuminate\Support\Facades\Log;
   
   // Example: Convert user input to ASCII (ignore unsupported chars)
   $userInput = "Café";
   $asciiOutput = iconv('UTF-8', 'ASCII//IGNORE', $userInput);
   Log::info("Converted: {$asciiOutput}"); // Logs "Converted: Cafe"
   

3. Verify Polyfill Activation: Simulate a missing ext-iconv environment:

   php -d extension=-iconv artisan tinker
   

   Run the same iconv() call in Tinker to confirm it works without native extension.

Implementation Patterns

Usage Patterns

1. Drop-in Replacement: Replace all iconv() calls with the polyfill’s native functions. No wrappers or adapters are needed:

   // Before/After: No code changes required
   $converted = iconv('UTF-8', 'ASCII//TRANSLIT', $string);
   

2. Environment-Specific Logic: Use the polyfill to handle edge cases where native iconv might fail:

   // Safely convert with fallback
   $result = @iconv('UTF-8', 'ASCII//IGNORE', $text) ?: $text;
   

3. File Handling: Sanitize filenames or content from non-Latin scripts:

   // Laravel File Upload Example
   $filename = $request->file('resume')->getClientOriginalName();
   $sanitized = iconv('UTF-8', 'ASCII//IGNORE', $filename);
   $request->file('resume')->move(storage_path('uploads'), $sanitized);
   

4. Database Integration: Normalize legacy encodings (e.g., ISO-8859-1) to UTF-8 for storage:

   // Laravel Eloquent Example
   $record->title = iconv('ISO-8859-1', 'UTF-8', $rawTitle);
   $record->save();
   

5. Testing: Force polyfill usage in tests to catch encoding issues early:

   // In phpunit.xml or test case setup
   putenv('PHP_EXT_ICONV=0'); // Simulate missing extension
   

Workflows

1. Multilingual Content Workflow:

   • Use polyfill for user-generated content (comments, profiles) in non-Latin scripts.
   • Pair with Laravel’s Str::of() for additional string manipulation:

     $cleanText = Str::of(iconv('UTF-8', 'ASCII//TRANSLIT', $userComment))->lower();
     

2. Legacy System Migration:

   • Gradually replace mbstring calls with iconv where consistency is critical.
   • Example: Replace mb_strlen() with iconv_strlen() for byte-length calculations.

3. CI/CD Pipeline:

   • Ensure tests pass in environments without ext-iconv:

     # GitHub Actions Example
     jobs:
       test:
         runs-on: ubuntu-latest
         steps:
           - run: php -d extension=-iconv vendor/bin/phpunit
     

Integration Tips

1. Laravel-Specific:

   • File Uploads: Use the polyfill in HandleUploadedFile or custom request handlers.
   • Validation: Extend Laravel’s FormRequest to sanitize input:

     protected function sanitizeInput($input) {
         return iconv('UTF-8', 'ASCII//IGNORE', $input);
     }
     

2. Performance Optimization:

   • Cache frequent conversions (e.g., static translations):

     $cacheKey = 'translation_' . md5($locale . $key);
     $translation = Cache::remember($cacheKey, now()->addHours(1), function() use ($locale, $key) {
         return iconv('UTF-8', 'ASCII//TRANSLIT', $rawTranslation);
     });
     

3. Monitoring:

   • Log polyfill usage to detect unexpected activations:

     if (!extension_loaded('iconv')) {
         Log::warning('Using symfony/polyfill-iconv; consider enabling ext-iconv in production.');
     }
     

Gotchas and Tips

Pitfalls

1. Unsupported Features:

   • ob_iconv_handler: Not polyfilled. Use mbstring output buffering instead:

     ob_start('mb_output_handler');
     

   • iconv_get_encoding(): Fall back to mb_internal_encoding():

     $encoding = mb_internal_encoding();
     

2. Performance Overhead:

   • Polyfill is 2–5x slower than native iconv. Profile with Blackfire or Xdebug to identify bottlenecks.
   • Avoid in high-frequency loops (e.g., real-time processing). Example:

     // Bad: Polyfill in tight loop
     foreach ($largeDataset as $item) {
         $converted = iconv('UTF-8', 'ASCII//IGNORE', $item->text);
     }
     

3. Edge Cases:

   • Complex Scripts: Thai, Arabic, or CJK transliteration (//TRANSLIT) may produce unexpected results. Test thoroughly.
   • Invalid Input: Polyfill may not handle malformed UTF-8 gracefully. Validate input first:

     if (!mb_check_encoding($text, 'UTF-8')) {
         $text = mb_convert_encoding($text, 'UTF-8', 'auto');
     }
     

4. False Positives:

   • Polyfill activates only when ext-iconv is missing. Ensure testing accounts for this:

     # Test with polyfill
     php -d extension=-iconv artisan test
     
     # Test with native extension (if available)
     php artisan test
     

Debugging

1. Verify Polyfill Activation:

   if (!extension_loaded('iconv')) {
       echo "Using polyfill: " . class_exists('Symfony\Polyfill\Iconv\Iconv') ? 'Yes' : 'No';
   }
   

2. Common Errors:

   • iconv() Warnings: Check for invalid character sets or malformed input.
   • Mojibake: Garbled text often indicates encoding mismatches. Example fix:

     $fixed = iconv('UTF-8', 'UTF-8//IGNORE', $garbledText);
     

3. Logging:

   • Log encoding operations to track issues:

     Log::debug("iconv('{$from}', '{$to}', '{$text}') => " . iconv($from, $to, $text));
     

Tips

1. Configuration:

   • No config/ or service provider setup is needed. The polyfill auto-registers via Composer.

2. Testing:

   • Use Laravel’s RefreshDatabase trait with polyfill-enabled tests:

     use Illuminate\Foundation\Testing\RefreshDatabase;
     
     class EncodingTest extends TestCase {
         use RefreshDatabase;
     
         public function testPolyfillWorks() {
             $this->assertEquals('Cafe', iconv('UTF-8', 'ASCII//IGNORE', 'Café'));
         }
     }
     

3. Extension Points:

   • Custom Polyfill: Extend Symfony\Polyfill\Iconv\Iconv for proprietary encoding schemes (rarely needed).
   • Feature Flags: Toggle polyfill usage for A/B testing:

     if (config('app.use_iconv_polyfill')) {
         // Force polyfill (for testing)
         if (extension_loaded('iconv')) {
             dl('iconv.so'); // Unload native extension (Linux)
         }
     }
     

4. Alternatives:

   • For mbstring-only projects, consider symfony/polyfill-mbstring instead.
   • For high-performance needs, enable ext-iconv in production and use the polyfill only in CI/staging.

5. Laravel-Specific:

   • File System: Override Illuminate\Filesystem\Filesystem to use polyfill for path sanitization.
   • Validation: Extend Illuminate\Validation\Validator to handle encoded input:

     $validator->extend('encoded', function ($attribute, $value, $parameters) {
         return iconv('UTF-8', 'ASCII//IGNORE', $value) === $value;
     });