helgesverre/toon
TOON is a compact, human-readable data format that cuts token usage when sending structured data to LLMs. This PHP port offers simple encode and decode methods, works on PHP 8.1+, and supports arrays, objects, and primitives.
[ or { — for example k: "a[3]b", or log lines carrying ANSI escape sequences such as "[31m…" — were misread as array headers and raised SyntaxException: Unterminated quoted string on decode. Header detection now locates the : separator and the [/{ markers outside quoted spans, so encoder output round-trips through Toon::decode() for every string value. Regression introduced in v3.2.0 (v3.1.0 was unaffected). Added comprehensive bidirectional round-trip test coverage (tests/Spec/RoundTripSpecialStringsTest.php, tests/Spec/ComprehensiveRoundTripTest.php).docs/SPEC.md and docs/CHANGELOG.md.\n, \r, \t) as \uXXXX; decoder accepts \uXXXX (case-insensitive hex) and rejects lone surrogates and escapes with fewer than four hex digits.[] and key: [] forms in addition to the legacy [0]: / key[0]: forms.Toon::validate(), toon_validate() and toon_validate_lenient() for checking TOON syntax without decoding.tests/Spec/Version31To33ComplianceTest.php.n = 0 or 1e-6 ≤ |n| < 1e21; values outside that range use exponent notation (lowercase e, explicit sign, e.g. 1e+21, 1e-7). Large in-domain floats are no longer quoted as strings, and very small numbers no longer underflow to 0.[03] (§6, §14.2), header delimiter mismatches where the bracket delimiter differs from the field-list delimiter, e.g. rows[2|]{a,b}: (§6, §14.2), and duplicate sibling keys at the same depth (§8, §14.4). Non-strict mode treats malformed bracket tokens as literal keys and applies last-write-wins for duplicate keys."App\\Controller::method") now decode correctly (#4, #5).key: [] previously decoded to null and a bare [] crashed with an uninitialized-offset warning; both now decode to an empty array.- [M]: … → inner array, - key: … → object, bare - → empty object. This fixes round-trips for arrays of arrays and lists of single-field objects. Nested arrays of objects/arrays as list items also round-trip now.[]) in both modes, instead of throwing (strict) or returning null (lenient).key: (§8): decodes to an empty object ([]), not null."my-key"[3]: is unescaped.[3]: a,,b → ['a','','b'])."a\\": c) parses.EncodeOptions/DecodeOptions reject indent < 1 (indent 0 cannot represent nesting); EncodeOptions::compact() now uses indent: 1 so nested output round-trips.Toon::validate() now runs the full decoder internally and returns false only when decode() throws, so validation and decoding can never disagree. The separate validator implementation (which had drifted from the decoder on duplicate keys, malformed brackets, and over-indented list fields) was removed.toJSON() method can now provide custom serialization, similar to JSON.stringify in JavaScript. The method takes priority over JsonSerializable interface and includes recursion protectiontests/Spec/Version3BreakingChangesTest.php with 6 tests verifying v3.0 compliancetest_tabular_first_field_in_list_item_round_trip to verify v3.0 format survives encode/decode cycles-05, -0001) as strings per TOON Specification §2.4This major release aligns with TOON Specification v2.0, which removes the optional # length marker prefix from array headers. The library version now matches the spec version for clarity.
EncodeOptions::$lengthMarker parameter - The optional length marker parameter has been removed from the constructorEncodeOptions::withLengthMarkers() preset - This preset method has been removedEncodeOptions::withLengthMarker() method - This fluent setter has been removed[N] format (e.g., [3]: a,b,c). The deprecated [#N] format is no longer supported.[#N] format with SyntaxException. Previously accepted both [N] and [#N] formats.Before (v1.x):
use HelgeSverre\Toon\EncodeOptions;
use HelgeSverre\Toon\Toon;
// Using preset with length markers
$options = EncodeOptions::withLengthMarkers();
$toon = Toon::encode($data, $options);
// Output: [#3]: a,b,c
// Using constructor
$options = new EncodeOptions(lengthMarker: '#');
$toon = Toon::encode($data, $options);
After (v2.0):
use HelgeSverre\Toon\EncodeOptions;
use HelgeSverre\Toon\Toon;
// Use default options (no length marker parameter)
$options = EncodeOptions::default();
$toon = Toon::encode($data, $options);
// Output: [3]: a,b,c
// Constructor no longer accepts lengthMarker
$options = new EncodeOptions();
$toon = Toon::encode($data, $options);
[#N] format[N] format[#N] format with clear error messageslengthMarker parameter and related methodsEncodeOptions::$lengthMarker propertyEncodeOptions::withLengthMarkers() static methodEncodeOptions::withLengthMarker() instance methodsprintf() calls that were locale-dependent, causing decimal points to become commas in locales like da_DK, de_DE, etc. Now uses number_format() with explicit decimal point separator for guaranteed locale-independent formatting per TOON Spec §2. This ensures output is always valid regardless of system locale settings.InvalidArgumentException. Per TOON Spec §7.1, only \n, \r, and \t have defined escape sequences. This prevents potential security issues from raw control characters in output.[N#] (hash after digits) now produces the correct v2.0 error message instead of a generic errortests/Spec/Version2BreakingChangesTest.php)docs/SPEC-COMPLIANCE.md with control character policy and v2.0 verification detailsjust benchmark-performance - Run PHPBench with default reportjust benchmark-perf-summary - Run with summary reportjust benchmark-all - Run both token efficiency and performance benchmarksjust benchmark-baseline - Store current performance as baselinejust benchmark-compare - Compare against stored baselineBackedEnum and UnitEnum types
Status::ACTIVE → "active")Counting::TWO → "TWO")HttpCode::cases() → "[2]: 201,400")[0] length marker (e.g., items[0]:)just commands for:
Toon::encode() method\t string in array headerstrue, false, null are quoted, not True, False, etc.encodeKey() method that uses identifier pattern matching (^[A-Za-z_][\w.]*$) instead of applying value quoting rules to keysjson_encode() and number_format() to avoid locale-dependent decimal separatorsJsonSerializable first, then toArray(), then public properties only (via get_object_vars()) to prevent leaking private/protected properties0xFF) and binary (0b1010) pattern detection to ensure they are properly quotedencodeArray() methodsetlocale() calls that could cause thread-safety issuesToon::encode() static methodEncodeOptions:
EncodeOptions with fluent APIHow can I help you explore Laravel packages today?