String Encode Laravel Package

Getting Started

Minimal Steps

1. Installation:

   composer require paquettg/string-encode
   

   Ensure ext-mbstring is enabled in your php.ini (required by Laravel).

2. First Use Case: Convert a multibyte string (e.g., user input) to UTF-8 in a Laravel controller:

   use StringEncoder\Encoder;
   
   public function processInput(Request $request) {
       $encoder = new Encoder();
       $cleanText = $encoder->convert()->fromString($request->input('text'))->toString();
       // $cleanText is now UTF-8 encoded
   }
   

3. Where to Look First:

   • README.md: Basic usage and installation.
   • docs/encoding.md: Encoding conversion workflows.
   • docs/regex.md: Multibyte regex patterns (critical for Laravel validation/localization).

Implementation Patterns

Usage Patterns

1. Fluent Conversion Workflow:

   // Convert ISO-8859-1 to UTF-8
   $encoder->convert()->fromString($str)->toUTF8();
   
   // Validate encoding before conversion
   $encoder->validate($str, 'UTF-8');
   

2. Laravel Integration:

   • Service Provider Registration:

     // app/Providers/AppServiceProvider.php
     public function register() {
         $this->app->singleton(Encoder::class, function () {
             return new Encoder(['defaultEncoding' => 'UTF-8']);
         });
     }
     

   • Facade for Convenience:

     // app/Facades/StringEncoder.php
     namespace App\Facades;
     use Illuminate\Support\Facades\Facade;
     class StringEncoder extends Facade {
         protected static function getFacadeAccessor() { return 'encoder'; }
     }
     

     Usage:

     StringEncoder::convert()->fromString($text)->toUTF8();
     

3. Validation in Form Requests:

   // app/Http/Requests/StorePostRequest.php
   use StringEncoder\Encoder;
   public function rules() {
       return [
           'title' => ['required', function ($attribute, $value, $fail) {
               $encoder = app(Encoder::class);
               if (!$encoder->validate($value, 'UTF-8')) {
                   $fail('The title must be valid UTF-8.');
               }
           }],
       ];
   }
   

4. Bulk Conversions (e.g., CSV Imports):

   $encoder = new Encoder();
   $rows = collect($csvData)->map(function ($row) use ($encoder) {
       return collect($row)->map(function ($cell) use ($encoder) {
           return $encoder->convert()->fromString($cell)->toUTF8();
       });
   });
   

5. Regex with Multibyte Strings:

   $encoder = new Encoder();
   $pattern = '/[^\p{L}]/u'; // Unicode-aware pattern
   $matches = $encoder->regex()->match($pattern, $text);
   

Workflows

1. Data Migration:

   • Normalize legacy database fields:

     DB::table('posts')->update([
         'title' => DB::raw('CONVERT(`title` USING utf8mb4)'),
     ]);
     

   • Use the package for validation:

     $encoder->validate($title, 'UTF-8') || throw new \Exception('Invalid encoding');
     

2. User-Generated Content:

   • Sanitize filenames/uploads:

     $cleanName = $encoder->convert()->fromString($request->file('avatar')->getClientOriginalName())->toAscii();
     

3. Localization:

   • Ensure translated strings are UTF-8:

     $translated = trans('messages.welcome');
     $encoder->validate($translated, 'UTF-8') || Log::error('Translation encoding issue');
     

Integration Tips

• Leverage Laravel’s Str Helper: Combine with Str::of() for hybrid operations:

  $cleanText = Str::of($text)->replaceMatches('/[^\p{L}]/u', '')
      ->toString();
  

• Database Collations: Use utf8mb4 in migrations for full Unicode support:

  Schema::create('posts', function (Blueprint $table) {
      $table->string('title')->collation('utf8mb4_unicode_ci');
  });
  

• Testing: Mock the Encoder in unit tests:

  $this->partialMock(Encoder::class, ['validate']);
  $encoder->shouldReceive('validate')->once()->andReturn(true);
  

Gotchas and Tips

Pitfalls

1. BOM Handling:

   • The removeBOM option may strip unexpected bytes (e.g., in uploaded files). Test with:

     $encoder = new Encoder(['removeBOM' => true]);
     

   • Tip: Disable for non-text files (e.g., images).

2. Encoding Detection:

   • mb_detect_encoding() is unreliable for mixed-encoding strings. Use mb_check_encoding() for validation:

     if (!mb_check_encoding($str, 'UTF-8')) {
         throw new \InvalidArgumentException('Invalid UTF-8 string');
     }
     

3. Regex Performance:

   • mb_regex is slower than PCRE. Benchmark in high-traffic routes:

     // Avoid in loops
     $encoder->regex()->match('/pattern/u', $text);
     

4. PHP 8.0+ Compatibility:

   • The package drops support for PHP <7.2 but may not be tested on PHP 8.0+. Verify:
     • No deprecated mb_* functions.
     • No Encode class (removed in v2.0.0).

5. File I/O:

   • Writing conversions to files (toFile()) may overwrite existing files. Use unique filenames:

     $encoder->convert()->fromString($text)->toFile(storage_path('app/encoded_'.uniqid().'.txt'));
     

Debugging

1. Invalid Encodings:

   • Check for mb_convert_encoding() warnings. Use mb_internal_encoding() to debug:

     mb_internal_encoding('UTF-8');
     $detected = mb_detect_encoding($str);
     

2. Facade Issues:

   • Ensure the Encoder is registered in the service container:

     php artisan vendor:publish --provider="StringEncoder\EncoderServiceProvider"
     

   • Tip: Use app(Encoder::class) directly if the facade fails.

3. Regex Failures:

   • Validate patterns with preg_last_error() or mb_regex_encoding():

     mb_regex_encoding('UTF-8');
     $matches = mb_ereg_match('/pattern/u', $text);
     

Config Quirks

1. Default Encoding:

   • Override globally in the service provider:

     $this->app->singleton(Encoder::class, function () {
         return new Encoder(['defaultEncoding' => 'Windows-1252']);
     });
     

2. Case-Sensitive Validation:

   • Enable for strict checks:

     $encoder = new Encoder(['caseSensitive' => true]);
     $encoder->validate($str, 'UTF-8');
     

3. Proxy Support:

   • Use static calls via the proxy:

     $result = \StringEncoder\Encoder::convert()->fromString($str)->toUTF8();
     

Extension Points

1. Custom Encoders:

   • Extend the Encoder class for domain-specific logic:

     class CustomEncoder extends Encoder {
         public function toAscii() {
             return $this->convert()->toString('ASCII', 'UTF-8');
         }
     }
     

2. Laravel Events:

   • Trigger encoding validation on model events:

     // app/Models/Post.php
     protected static function booted() {
         static::saving(function ($post) {
             $encoder = app(Encoder::class);
             if (!$encoder->validate($post->title, 'UTF-8')) {
                 throw new \Exception('Title must be UTF-8 encoded');
             }
         });
     }
     

3. Middleware:

   • Enforce encoding in API requests:

     // app/Http/Middleware/EncodeRequest.php
     public function handle($request, Closure $next) {
         $encoder = app(Encoder::class);
         foreach ($request->all() as $key => $value) {
             if (!$encoder->validate($value, 'UTF-8')) {
                 return response('Invalid encoding', 400);
             }
         }
         return $next($request);
     }