Php Unicode Laravel Package

Getting Started

Install and confirm platform requirements.

composer require cybercog/php-unicode

The package requires PHP 8.1+ and ext-mbstring. ext-intl is optional but mandatory if you want the grapheme-level classes (Grapheme, GraphemeString). Add both to your own composer.json require block so a missing extension fails loudly at install time rather than at runtime on a production box:

"require": {
    "php": "^8.1",
    "ext-mbstring": "*",
    "ext-intl": "*",
    "cybercog/php-unicode": "^1.0"
}

Pin an exact or tilde version — this package has a single star, zero dependents, and a very recent release. Vendor review and a locked version are non-negotiable for production.

Read the source before writing code against it. The documented surface is tiny, and the four classes in src/ are the whole library. Read these in order:

1. src/CodePoint.php — the atomic unit; factory methods of(), ofDecimal(), ofHexadecimal(), ofHtmlEntity(), ofXmlEntity(), and the decimal/hex/HTML/XML conversion outputs.
2. src/UnicodeString.php — a sequence of CodePoint objects, built with of() / ofCodePointList().
3. src/Grapheme.php and src/GraphemeString.php — same idea at user-perceived-character level, backed by grapheme_*.

Heads up on the docs: the package blurb advertises a fluent StringObject API wrapping mbstring/iconv for length/substring/case/trim/pad. The actual architecture described in AGENTS.md is four final immutable value objects with private constructors and static factories — a code-point/grapheme toolkit, not a fluent string wrapper. Trust the source tree over the description. If you need the fluent API, symfony/string already covers it; this package is best treated as character-level inspection and validation.

First use case — grapheme-aware length validation. This is the highest-value thing you can do on day one, because Laravel's built-in max rule uses mb_strlen, which counts code points, not user-perceived characters. A family emoji (👨‍👩‍👧‍👦) is 7 code points, a flag is 2, a skin-toned emoji is 2. Users hit max:20 at what looks like 9 characters.

namespace App\Rules;

use Closure;
use Cog\Unicode\GraphemeString;
use Illuminate\Contracts\Validation\ValidationRule;

class GraphemeLength implements ValidationRule
{
    public function __construct(private int $max) {}

    public function validate(string $attribute, mixed $value, Closure $fail): void
    {
        if (!is_string($value)) {
            return;
        }

        // verify the count accessor: Countable interface or ->count()
        if (count(GraphemeString::of($value)) > $this->max) {
            $fail("The {$attribute} field may not exceed {$this->max} characters.");
        }
    }
}

Then in a Form Request:

public function rules(): array
{
    return [
        'display_name' => ['required', 'string', new GraphemeLength(24)],
    ];
}

Implementation Patterns

Treat the classes as value objects, not strings. Every class is final, immutable, and has a private constructor. You never new one, and you never mutate one. UnicodeString::of('...') and GraphemeString::of('...') are your entry points; every "operation" produces a new instance or a scalar. Build a mental model of string → GraphemeString → list<Grapheme> → list<CodePoint> and only descend as far as you need.

Decide early: code point or grapheme?

Default to GraphemeString for anything user-facing. Reserve UnicodeString/CodePoint for analysis and security work where you want to see the raw code points.

Wrap it in a thin application service. Static factories are hostile to testing — you can't mock GraphemeString::of(). If you need to stub behaviour in tests, or if you want an escape hatch to swap in symfony/string later, put an interface and adapter between your domain and the package:

namespace App\Support\Text;

interface TextMeasurer
{
    public function graphemeLength(string $value): int;
    /** @return list<string> */
    public function graphemes(string $value): array;
}

Then implement it with Cog\Unicode\GraphemeString. Bind it in a service provider. Your domain code now has one dependency to swap.

Eloquent attribute cast for stored unicode fields. If you want model attributes to surface as UnicodeString instances rather than raw strings:

namespace App\Casts;

use Cog\Unicode\UnicodeString;
use Illuminate\Contracts\Database\Eloquent\CastsAttributes;
use Illuminate\Database\Eloquent\Model;

class UnicodeStringCast implements CastsAttributes
{
    public function get(Model $model, string $key, mixed $value, array $attributes): ?UnicodeString
    {
        return $value === null ? null : UnicodeString::of((string) $value);
    }

    public function set(Model $model, string $key, mixed $value, array $attributes): array
    {
        return [$key => $value instanceof UnicodeString ? (string) $value : $value];
    }
}

Use this sparingly. A cast means every attribute read pays the materialisation cost, and objects leak through to Blade/JSON unless you serialise carefully. For most apps, keep the database column a plain string and convert in the validation layer + API resource layer only.

Serialise back to plain strings at the boundary. UnicodeString and GraphemeString are internal domain types. In JsonResource::toArray(), Blade output, and API response bodies, convert to a primitive string first. Check whether the class implements __toString() and/or JsonSerializable — if not, add a ->toNative()/(string) call in your resource class. Do not assume json_encode on the object "just works."

Security filtering: invisible and bidi characters. This is where UnicodeString+CodePoint genuinely beats plain mb_* functions. You can spot zero-width, joiner, and RTL-override characters that enable username spoofing:

use Cog\Unicode\UnicodeString;

$blocklist = [0x200B, 0x200C, 0x200D, 0x200E, 0x200F, 0x202A, 0x202E, 0x2066, 0x2069];

foreach (UnicodeString::of($username) as $codePoint) {
    if (in_array($codePoint->toDecimal(), $blocklist, true)) {
        // flag for review instead of silently stripping
    }
}

Run this in a Rule on username, display name, and bio fields.

Testing pattern. Mirror the upstream style — PHPUnit with #[DataProvider] for input/expected pairs. Because the classes are pure value objects, they're trivially testable:

#[DataProvider('graphemes')]
public function test_counts_user_perceived_characters(string $input, int $expected): void
{
    $this->assertSame($expected, count(GraphemeString::of($input)));
}

public static function graphemes(): array
{
    return [
        'ascii'        => ['hello', 5],
        'single emoji' => ['👍', 1],
        'flag'         => ['🇺🇦', 1],
        'family'       => ['👨‍👩‍👧‍👦', 1],
        'skin tone'    => ['👍🏽', 1],
    ];
}

That test table alone justifies adopting the package over strlen/mb_strlen.

Gotchas and Tips

The advertised API and the real API may not match. The README blurb sells mbstring/iconv wrapping with a fluent StringObject; AGENTS.md describes immutable CodePoint/UnicodeString/Grapheme/GraphemeString. Before committing to a design, run a 10-line probe script and confirm which methods actually exist. Do not write fifty call sites against methods described only in marketing copy.

Code points vs graphemes is the #1 bug source. UnicodeString::of('👨‍👩‍👧‍👦') yields 7 code points. GraphemeString::of(...) yields 1. If you validate length with UnicodeString, you will reject valid names and truncate emoji mid-sequence. Conversely, if you scan for malicious characters with GraphemeString, you'll see an opaque cluster and miss the embedded ZWJ. Use both, for different jobs.

ext-intl is optional upstream — make it mandatory for you. Many minimal Docker images and CI images ship without intl. The moment your code calls GraphemeString::of(), it fatals on an undocumented class. Add ext-intl to composer.json require and add platform requirements in your CI matrix. Verify php -m | grep intl on every environment, including Laravel Octane workers and queue workers.

Performance: do not materialise megabyte strings. Each CodePoint/Grapheme is a heap-allocated object. A 1 MB UTF-8 string becomes roughly 1M objects if you materialise the whole thing — tens of megabytes of RAM and a lot of GC pressure. Rules of thumb:

• Use these classes on bounded inputs: single validation fields, names, subjects, short snippets.
• For bulk text (article bodies, search indexing, log lines), stay on native mb_* / grapheme_* functions.
• If you must measure a large string, consider checking strlen($value) < someThreshold with native functions first, and only escalate to Unicode objects for the expensive part.

Invalid UTF-8 is likely to throw, not degrade. UnicodeString::of() almost certainly validates and throws on malformed bytes. In Laravel this means an exception from a request payload can become a 500 instead of a 422. Guard your validation rules: run mb_check_encoding($value, 'UTF-8') (or Laravel's string rule) before constructing any Unicode object, and return false from the rule rather than letting an exception escape. Test with a deliberately broken byte sequence.

final + private constructors means no subclassing and no mocking. You cannot extend CodePoint to add behaviour, and you cannot inject a fake. Two consequences: (1) build around it with an adapter interface, as shown above; (2) any "add a method" instinct should become an application service instead. This is an intentional design choice, not an oversight — work with it.

Equality and identity. Two UnicodeString::of('café') instances (one NFC-composed é, one NFD e + combining accent) contain different code points and are not equal, even though they render similarly. If you need canonical comparison, normalise with Normalizer::normalize($value, Normalizer::FORM_C) from ext-intl before constructing the Unicode object. The package's own comparison utilities won't save you from un-normalised input — and if its comparison helpers are absent entirely, fall back to hash_equals/=== on normalised native strings.

HTML/XML entity round-tripping. CodePoint::ofHtmlEntity() / ofXmlEntity() are useful, but be aware that numeric entities and named entities differ, and htmlspecialchars in your Blade views already handles escaping. Don't stack the two — you'll double-encode. Use the entity methods for decoding untrusted incoming entity strings and for controlled output generation, not in addition to Laravel's default escaping.

No service provider, no config, no facade. There's nothing to auto-discover. No config/unicode.php exists. If you want a configurable blocklist of forbidden characters, put it in your own config/, not in a vendor override. Extension points are effectively zero (final classes) — the extension point is your own adapter layer.

Pin and monitor. Single-star, zero-dependent, freshly released, MIT. The upside is the source is ~4 files and auditable in fifteen minutes — do it. Add the package to your dependency-review checklist and watch for the release cadence before treating it as infrastructure. Keep your adapter interface around so the swap to symfony/string's grapheme_* methods (or a small internal helper) is a single-file change.

Debugging invisible characters. When a string "looks the same" but fails validation, dump the code points as hex:

foreach (UnicodeString::of($input) as $cp) {
    dump($cp->toHexadecimal(), $cp->toDecimal());
}

This turns invisible-character bugs (zero-width spaces pasted from a CMS, non-breaking spaces, BOMs) into obvious ones in seconds.