Detect Encoding Laravel Package

## Integration Approach

### **Migration Path**
   - **Phase 2: Wrapper Abstraction** (continued)
     ```php
         public function detect(string $text): string {
             if ($this->isWhitelistedEncoding($text)) {
                 return (new \Onnov\DetectEncoding\EncodingDetector())->getEncoding($text);
             }
             return mb_detect_encoding($text, $this->fallbackEncodings, true);
         }
     }
     ```
   - **Phase 3: Rollout**
     - Replace calls in **controllers**, **commands**, and **jobs** using a **regex search/replace** (e.g., `mb_detect_encoding(` → `app('encodingDetector')->detect(`).
     - Use **Laravel’s `app()` helper** for dependency injection:
       ```php
       $encoding = app(\App\Services\EncodingDetector::class)->detect($text);
       ```

### **Compatibility**
- **PHP Version**: Test on **PHP 7.4+** (Laravel’s LTS support). PHP 8.x may require minor adjustments (e.g., named arguments).
- **Laravel Version**: Compatible with **Laravel 7+** (no framework-specific code). For older versions, ensure `composer.json` constraints allow PHP 7.4+.
- **Dependency Conflicts**: None (single class, no Composer dependencies beyond PHP).
- **Encoding Support**:
  - **Enabled by Default**: `windows-1251`, `koi8-r`, `iso-8859-5`, `ibm866`.
  - **Disabled by Default**: `MAC_CYRILLIC` (enable only if needed).
  - **Custom Encodings**: Extend via `addEncoding()` if supporting niche cases (e.g., `x-mac-cyrillic`).

### **Sequencing**
1. **Add to `composer.json`**:
   ```bash
   composer require onnov/detect-encoding

2. Create Service Class:

   php artisan make:service EncodingDetector
   

3. Register Service:
   • In AppServiceProvider@boot():

     $this->app->singleton(\App\Services\EncodingDetector::class);
     

   • Or use autowiring (Laravel 8+):

     // app/Providers/AppServiceProvider.php
     public function register()
     {
         $this->app->bind(\App\Services\EncodingDetector::class);
     }
     

4. Pilot Testing:
   • Test with known-encoding samples (e.g., UTF-8 vs. windows-1251).
   • Log false positives/negatives for 1 week in production.
5. Gradual Replacement:
   • Start with non-critical paths (e.g., admin imports).
   • Monitor performance impact (e.g., tideways or blackfire.io).
6. Document:
   • Add to API docs if used in public endpoints.
   • Note supported encodings and fallback behavior in README.md.

Operational Impact

Maintenance

• Low Effort:
  • No external API calls or scheduled tasks.
  • Single class to monitor for updates (though inactive since 2021).
• Upgrade Path:
  • If PHP 8.x breaks compatibility, fork and patch (minimal changes expected).
  • Alternative: Migrate to Symfony’s StringUtils or PHP’s intl extension if this package stagnates.
• Deprecation:
  • Phase out if Laravel’s mb_detect_encoding improves (track PHP RFCs).

Support

• Debugging:
  • Logging: Add debug logs for misdetections:

    \Log::debug('Encoding detection failed', ['text' => substr($text, 0, 100), 'result' => $encoding]);
    

  • Fallback Transparency: Alert users when falling back to mb_detect_encoding.
• User Impact:
  • Data Corruption Risk: If misconfigured, could silently corrupt text. Mitigate with:
    • Validation: Reject texts with undetected encodings in critical paths.
    • UI Warnings: Flag files with non-UTF-8 encodings (e.g., "This file uses Windows-1251 encoding").

Scaling

• Performance:
  • Microbenchmark: Test with 1M+ characters (README claims 0.00096s for 1.3M Cyrillic chars).
  • Caching: Cache results for identical texts (e.g., in Redis) if used in loops.
  • Bottlenecks: Avoid in hot loops (e.g., real-time validation). Use for batch processing instead.
• Concurrency:
  • Stateless class; safe for parallel processing (e.g., Laravel queues).
• Resource Usage:
  • Memory: Minimal (no external calls).
  • CPU: Lightweight; no blocking I/O.

Failure Modes

Ramp-Up

• Onboarding:
  • Documentation: Add to Laravel’s internal or docs/encoding.md.
  • Examples:

    // Detect encoding
    $encoding = app(\App\Services\EncodingDetector::class)->detect($text);
    
    // Convert to UTF-8
    $utf8Text = app(\App\Services\EncodingDetector::class)->toUtf8($text);
    

• Training:
  • Dev Workshop: Demo accuracy vs. mb_detect_encoding with real-world samples.
  • Checklist: Provide a pre-migration validation script to compare outputs.
• Adoption Metrics:
  • Track usage in PRs (e.g., "X% of encoding detections now use the new package").
  • Monitor false-positive rates post-launch.
• Rollback Plan:
  • Feature Flag: Toggle between old/new detectors.
  • Database Backup: For bulk migrations using the package.

```markdown
## Operational Impact (Continued)

### **Monitoring**
- **Key Metrics**:
  - **Detection Accuracy**: Log success/failure rates by encoding (e.g., Prometheus counter).
  - **Performance**: Track detection time for large texts (e.g., `histogram` in APM tools).
  - **Fallback Usage**: Percentage of detections using `mb_detect_encoding` (should trend to 0%).
- **Alerts**:
  - **Anomaly Detection**: Alert if false-positive rate exceeds 1% for a given encoding.
  - **Latency Spikes**: Notify if detection time > 10ms for texts >10KB.

### **Disaster Recovery**
- **Data Corruption**:
  - **Audit Trail**: Log original text + detected encoding for reversible changes.
  - **Backup**: Store raw files alongside processed data during migrations.
- **Outage**:
  - **Graceful Degradation**: Fall back to `mb_detect_encoding` or disable encoding detection entirely (with user notification).

### **Team Skills**
- **Required**:
  - Familiarity with **Laravel service containers** and **dependency injection**.
  - Basic **PHP string manipulation** (e.g., `iconv`, `mb_*` functions).
- **Upskill**:
  - **Encoding Theory**: Train team on `windows-1251` vs. `koi8-r` differences (critical for debugging).
  - **Performance Profiling**: Use tools like **Xdebug** to analyze detection bottlenecks.

### **Cost Analysis**
- **Development**:
  - **Low**: ~5–10 dev hours for integration (wrapper + tests).
- **Operational**:
  - **None**: No hosting, APIs, or licenses.
- **Risk Mitigation**:
  - **High**: Potential data corruption if misconfigured (offset by fallback logic).
  - **Low**: Maintenance or scaling costs.

### **Stakeholder Communication**
- **For Developers**:
  - *"Use `app('encodingDetector')->detect($text)` instead of `mb_detect_encoding`. For custom encodings, extend the service class."*
- **For QA**:
  - *"Test with Cyrillic/Windows-1251 texts. Log any misdetections for review."*
- **For Product**: