Getting Started

ddeboer/transcoder is a thin abstraction over PHP's three encoding-conversion backends — iconv, mbstring, and GNU recode — so you can write one call site and let the library pick whichever extension is actually installed. It is a single-purpose library: it converts a string from one character set to another, and throws a dedicated exception when it can't.

Install

composer require ddeboer/transcoder

There are no Laravel service providers to publish and no config file. The only real requirement is that at least one of ext-iconv or ext-mbstring is enabled — in practice both are, on any modern PHP build.

Where to look first

src/Transcoder.php — the Transcoder::create() factory. This is the entry point; it probes the runtime for available extensions and returns a concrete transcoder.
TranscoderInterface::transcode($string, $sourceEncoding, $targetEncoding) — the one method you'll call 95% of the time.
The per-backend classes (IconvTranscoder, MbstringTranscoder, RecodeTranscoder) — read these when you need to understand why a particular string failed, because each backend behaves differently on malformed input.

First use case

You inherit a customers table or a nightly supplier feed encoded in Windows-1252/ISO-8859-1, and accented names arrive as Ã©, Ã¼, â€™. The fix is a single transcode before the value ever touches Eloquent:

use Ddeboer\Transcoder\Transcoder;

$transcoder = Transcoder::create();

$utf8 = $transcoder->transcode($row['name'], 'Windows-1252', 'UTF-8');
// "Müller" instead of "MÃ¼ller"

Bind it once in a service provider rather than calling Transcoder::create() in every request — the factory does extension detection on each call:

// app/Providers/AppServiceProvider.php
use Ddeboer\Transcoder\Transcoder;
use Ddeboer\Transcoder\TranscoderInterface;

public function register(): void
{
    $this->app->singleton(TranscoderInterface::class, fn () => Transcoder::create());
}

Then inject TranscoderInterface anywhere you need it.

Implementation Patterns

Legacy CSV import (the canonical use case). Transcode line-by-line while streaming, so a 200 MB supplier file doesn't land in memory:

use Ddeboer\Transcoder\TranscoderInterface;

final class ImportCustomers
{
    public function __construct(private TranscoderInterface $transcoder) {}

    public function handle(string $path): void
    {
        $file = new \SplFileObject($path, 'r');

        foreach ($file as $line) {
            if (trim($line) === '') {
                continue;
            }

            $utf8 = $this->transcoder->transcode($line, 'Windows-1252', 'UTF-8');
            $row  = str_getcsv($utf8, ';');

            Customer::updateOrCreate(
                ['external_id' => $row[0]],
                ['name' => $row[1], 'city' => $row[2]],
            );
        }
    }
}

Wrap the whole thing in a queued job with ShouldBeUnique if imports can overlap — transcoding is cheap, but re-running a half-finished import is not.

Transcode at the boundary, not in the model. Convert in the importer, the form request, or the API client — never inside an Eloquent accessor. Accessors run on every read, including reads of data that is already UTF-8, and you'll end up debugging double-encoded rows six months later. The database connection should be utf8mb4; the transcoder's job ends the moment the string reaches the model.

Legacy form posts / webhook payloads. A middleware or FormRequest prepareForValidation() hook handles the case where a third-party system POSTs ISO-8859-1:

protected function prepareForValidation(): void
{
    $this->merge(collect($this->all())->map(
        fn ($value) => is_string($value)
            ? $this->transcoder->transcode($value, 'ISO-8859-1', 'UTF-8')
            : $value
    )->all());
}

Guard this behind a header or a route group — you do not want it on modern JSON endpoints.

One-off remediation command. For data already sitting in the database in the wrong encoding, an Artisan command with chunking is the cleanest escape hatch:

Customer::query()->chunkById(500, function ($customers) {
    foreach ($customers as $customer) {
        $customer->name = $this->transcoder->transcode($customer->name, 'Windows-1252', 'UTF-8');
        $customer->saveQuietly(); // skip observers/events during remediation
    }
});

Write the original values to a backup column (or a JSON audit log) first. Mojibake repair is easy to get wrong in one direction and irreversible in the other.

Encoding detection pipeline. Don't ask the transcoder to guess. Detect with a strict candidate list, then hand the result to the transcoder:

$encoding = mb_detect_encoding($string, ['UTF-8', 'Windows-1252', 'ISO-8859-1'], true)
    ?? throw new \RuntimeException('Unknown encoding');

if ($encoding !== 'UTF-8') {
    $string = $this->transcoder->transcode($string, $encoding, 'UTF-8');
}

The order matters: ISO-8859-1 is a strict subset of Windows-1252 for most bytes, but the 0x80–0x9F range differs, and that range is exactly where smart quotes and em-dashes live. Listing Windows-1252 first gives you better results on real-world Word/Excel exports.

Backend pinning for consistency. The factory auto-detects, which means dev, CI, and production can silently end up on different backends with different error behaviour. Pin the backend explicitly in config (check the Transcoder::create() signature in vendor/ for the exact option key in your installed version) so tests and production agree. If you need full control, implement TranscoderInterface yourself and bind that — it's a one-method interface, so wrapping symfony/string's AbstractString or Patchwork\Utf8 is trivial and gives you a supported dependency path.

Testing. Keep a fixture directory of byte-exact files: a Windows-1252 CSV, an ISO-8859-1 export with ü/é, and a UTF-8 file with a BOM. Assert on the resulting UTF-8 strings. Round-trip tests (UTF-8 → ISO-8859-1 → UTF-8) catch lossy conversions early — anything outside Latin-1 will not survive, and you want that to fail in CI, not in a support ticket.

Gotchas and Tips

This package is effectively frozen. Last release was January 2015. Verify it installs cleanly on your PHP version before committing to it, and check whether recode support still compiles — the recode extension was removed from PHP core in 7.4, so on any current runtime that backend is unavailable and the factory will fall through to iconv/mbstring. If you're on PHP 8.1+, budget an afternoon to confirm the whole path works, or plan to replace it. Given the surface area (one interface, one method), a hand-rolled wrapper over mb_convert_encoding() is a legitimate alternative — the package's real value is the backend abstraction and the exception type, not the conversion logic.

iconv truncates silently. Without special handling, iconv stops at the first byte it can't map and returns a partial string with no error. If you see truncated values at the end of imported rows, that's this. mbstring behaves differently — it substitutes ? or U+FFFD depending on mb_substitute_character. Same input, different corruption, depending on which backend the factory picked. This is the single strongest argument for pinning the backend.

Characters that can't be represented are lost, not transliterated. Converting UTF-8 → ISO-8859-1 will drop or mangle €, …, „, and anything Cyrillic or CJK. If you need best-effort output, iconv's //TRANSLIT and //IGNORE suffixes are the usual escape hatch — check whether your installed transcoder passes them through; if not, drop to iconv() directly for that specific call.

Strip the BOM before transcoding. A UTF-8 BOM (EF BB BF) will survive conversion and show up as ï»¿ at the start of your first CSV column, which then fails validation with a baffling error. preg_replace('/^\xEF\xBB\xBF/', '', $line) on the first line only.

Never transcode twice. UTF-8 → UTF-8 is a no-op, so it's tempting to leave a transcode call in place "just in case" after you've fixed the source. Then someone fixes the upstream feed, and you're decoding already-correct data into garbage. Make the encoding an explicit, per-source config value and remove the call when the source is fixed.

mb_detect_encoding is unreliable on short strings. It cannot distinguish ISO-8859-1 from Windows-1252 from UTF-8 on a three-character surname. If you must auto-detect, require strict: true, pass an ordered candidate list, and have a fallback — but prefer a configured per-integration encoding, because you usually know the answer.

Catch the exception type, not \Exception. The library throws its own exception class on conversion failure. Catching broadly around an import will also swallow your QueryException and leave half-written rows. Let database errors bubble; catch only the transcoder's exception and decide per-row whether to skip, log, or abort.

Extension points are minimal by design. There's no config, no events, no middleware. The extension surface is: implement TranscoderInterface for a custom backend, or swap the singleton binding for a decorator that adds logging/metrics. The latter is worth doing on any ingestion pipeline — log source, target, backend, and input length on failure, because "conversion failed on row 4,127" with no encoding context is not a debuggable error.

Transcoder Laravel Package

Getting Started

Implementation Patterns

Gotchas and Tips