helgesverre/toon
TOON is a compact, human-readable data format that cuts token usage when sending structured data to LLMs. This PHP port offers simple encode and decode methods, works on PHP 8.1+, and supports arrays, objects, and primitives.
Patch release fixing a decoder round-trip regression introduced in v3.2.0.
[ or { — for example k: "a[3]b", or log lines carrying ANSI escape sequences such as "[31m…" — were misread as array headers and raised SyntaxException: Unterminated quoted string on decode, so the encoder's own output could not be decoded back. Header detection now locates the : separator and the [/{ markers outside quoted spans, so encoder output round-trips through Toon::decode() for every string value. The regression was introduced in v3.2.0; v3.1.0 was unaffected.tests/Spec/RoundTripSpecialStringsTest.php, tests/Spec/ComprehensiveRoundTripTest.php, tests/Spec/MalformedHeaderDecodeTest.php); line coverage raised from 90% to 96%.DelimiterParser helpers) with no change to the public decode path.Full Changelog: https://github.com/HelgeSverre/toon-php/compare/v3.2.0...v3.2.1
This release brings the library into conformance with TOON Specification v3.3 (upstream advanced v3.0 → v3.3) and adds a validation API.
docs/SPEC.md and docs/CHANGELOG.md.\n, \r, \t) as \uXXXX; the decoder accepts \uXXXX (case-insensitive hex) and rejects lone surrogates and escapes with fewer than four hex digits.[] and key: [] forms in addition to the legacy [0]: / key[0]: forms.Toon::validate(), toon_validate() and toon_validate_lenient() for checking TOON syntax without decoding.tests/Spec/Version31To33ComplianceTest.php.n = 0 or 1e-6 ≤ |n| < 1e21; values outside that range use exponent notation (lowercase e, explicit sign, e.g. 1e+21, 1e-7). Large in-domain floats are no longer quoted as strings, and very small numbers no longer underflow to 0.[03] (§6, §14.2), header delimiter mismatches where the bracket delimiter differs from the field-list delimiter, e.g. rows[2|]{a,b}: (§6, §14.2), and duplicate sibling keys at the same depth (§8, §14.4). Non-strict mode treats malformed bracket tokens as literal keys and applies last-write-wins for duplicate keys."App\Controller::method") now decode correctly (#4, #5).key: [] previously decoded to null and a bare [] crashed with an uninitialized-offset warning; both now decode to an empty array.- [M]: … → inner array, - key: … → object, bare - → empty object. This fixes round-trips for arrays of arrays and lists of single-field objects.null (lenient).key: (§8): decodes to an empty object, not null."my-key"[3]: is unescaped.[3]: a,,b → ['a','','b'])."a\\": c) parses.EncodeOptions/DecodeOptions reject indent < 1; EncodeOptions::compact() now uses indent: 1 so nested output round-trips.Toon::validate() now runs the full decoder internally and returns false only when decode() throws, so validation and decoding can never disagree. The separate validator implementation was removed.Full Changelog: https://github.com/HelgeSverre/toon-php/compare/v3.1.0...v3.2.0
toJSON() method can now provide custom serialization, similar to JSON.stringify in JavaScript-05, -0001) as strings per spec §2.4This major release aligns with TOON Specification v3.0, which standardizes the encoding for list-item objects whose first field is a tabular array.
When a list-item object has a tabular array as its first field:
Before (v2.x):
items[1]:
- users[2]{id,name}:
1,Ada
2,Bob
status: active
After (v3.0):
items[1]:
- users[2]{id,name}:
1,Ada
2,Bob
status: active
tests/Spec/Version3BreakingChangesTest.php with 6 tests verifying v3.0 complianceThis major release aligns with TOON Specification v2.0, which removes the optional # length marker prefix from array headers.
EncodeOptions::$lengthMarker parameter removed from constructorEncodeOptions::withLengthMarkers() preset method removedEncodeOptions::withLengthMarker() fluent setter removed[N] format (e.g., [3]: a,b,c)[#N] format with SyntaxExceptionBefore (v1.x):
$options = EncodeOptions::withLengthMarkers();
$toon = Toon::encode($data, $options);
// Output: [#3]: a,b,c
After (v2.0):
$options = EncodeOptions::default();
$toon = Toon::encode($data, $options);
// Output: [3]: a,b,c
sprintf() calls that caused decimal points to become commas in locales like da_DK, de_DE. Now uses number_format() with explicit decimal separator for locale-independent formatting per TOON Spec §2.InvalidArgumentException. Only \n, \r, and \t have defined escape sequences per TOON Spec §7.1.[N#] now produces correct v2.0 error messagedocs/SPEC-COMPLIANCE.md with control character policy and v2.0 compliance detailsjust benchmark-performance - Run PHPBench with default reportjust benchmark-perf-summary - Run with summary reportjust benchmark-all - Run both token efficiency and performance benchmarksjust benchmark-baseline - Store current performance as baselinejust benchmark-compare - Compare against stored baselineNative PHP enum support for both BackedEnum and UnitEnum types:
Thanks to @AmolKumarGupta for contributing this feature!
[0] length marker
items:items[0]:Full changelog: https://github.com/HelgeSverre/toon-php/blob/main/CHANGELOG.md#120---2025-10-28
Token savings compared to other formats:
Full changelog: https://github.com/HelgeSverre/toon-php/blob/main/CHANGELOG.md#110---2025-10-28
TOON (Token-Oriented Object Notation) is a compact data format for reducing token consumption when sending structured data to Large Language Models.
composer require helgesverre/toon
use HelgeSverre\Toon\Toon;
echo Toon::encode([
'items' => [
['sku' => 'A1', 'qty' => 2, 'price' => 9.99],
['sku' => 'B2', 'qty' => 1, 'price' => 14.5]
]
]);
Output:
items[2]{sku,qty,price}:
A1,2,9.99
B2,1,14.5
Full changelog: https://github.com/HelgeSverre/toon-php/commits/v1.0.0
How can I help you explore Laravel packages today?