Portable Utf8 Laravel Package

Getting Started

Install via composer require voku/portable-utf8. Start by replacing fragile string operations with drop-in utf8_*() functions (e.g., utf8_strlen(), utf8_substr()) to avoid mbstring dependency checks. First use case: sanitize user-submitted text in a registration flow—use UTF8::clean($input) to strip BOM, null bytes, and invalid UTF-8 sequences before validation.

Implementation Patterns

• Middleware sanitization: Add a global request middleware (SanitizeUnicodeInput) that runs UTF8::clean() and utf8_to_ascii() on specific fields (e.g., username, email).
• Fallback-aware utility wrappers: Build lightweight helper functions that auto-select best available engine:

  function safe_strlen(string $str): int {
      return function_exists('mb_strlen') ? mb_strlen($str) : utf8_strlen($str);
  }
  

• Unicode-safe file input/output: When reading CSVs or logs from external systems, validate with utf8_is_valid() and normalize with UTF8::nfd() or UTF8::nfc() to prevent comparison mismatches.
• Fluent data normalization pipelines: Chain operations in domain services:
  u($name)->trim()->toLowerCase()->toAscii()->substr(0, 50)->toString().
• Emoji safety: Use utf8_is_valid() before storing user input containing emojis, and avoid filter_var($str, FILTER_SANITIZE_STRING) which corrupts multibyte chars.

Gotchas and Tips

• Confusing function names: utf8_encode()/utf8_decode() in this package ≠ PHP’s native ISO-8859-1 ↔ UTF-8 converters—they handle UTF-8 validation and safe conversion only. Never use them for arbitrary encoding conversion (e.g., ISO-8859-1 → UTF-8). Prefer symfony/polyfill-iconv or iconv for that.
• Avoid in hot paths: While optimized, utf8_* functions can be 1.5–3× slower than native mbstring. Use in I/O boundaries, not tight loops—benchmark with microtime(true) if performance-critical.
• Case sensitivity edge cases: Turkish ‘İ’/‘ı’ and German ‘ß’ behave differently under Unicode case-folding—avoid locale-specific calls unless explicitly calling utf8_strtolower($str, 'tr_TR') (which this library does support but rarely needed).
• Database warnings: MySQL’s utf8 (not utf8mb4) stores only up to 3-byte chars—use utf8_is_valid() and validate against schema before insert. Prefer Laravel’s utf8mb4_unicode_ci collations and utf8mb4 column types.
• Fallback handlers: When mbstring is missing and you need to decode broken data, register a custom fallback:

  UTF8::setFallbackHandler(function($str) { return @iconv('UTF-8', 'UTF-8//IGNORE', $str); });
  

• Debugging tip: Use UTF8::error_get_last() to inspect internal warnings after failing operations (e.g., invalid UTF-8 sequences causing silent utf8_substr() truncation).