yethee/tiktoken
PHP port of OpenAI tiktoken for fast tokenization. Get encoders by model or encoding name, encode text to token IDs, with default vocab caching and configurable cache dir. Optional experimental FFI lib mode (tiktoken-rs) for better performance on larger texts.
|
Package
|
Constraint
|
|---|---|
| symfony/service-contracts | ^2.5 || ^3.0 |
|
Package
|
Constraint
|
|---|---|
| doctrine/coding-standard | ^13 |
| mikey179/vfsstream | ^1.6.11 |
| phpbench/phpbench | ^1.2 |
| phpunit/phpunit | ^10.5.45 |
| psalm/plugin-phpunit | ^0.19.2 |
| vimeo/psalm | 6.15.1 |
How can I help you explore Laravel packages today?